No Retractions, Only Corrections: A manifesto.

Under the heading, “Why that Evolution paper should never have been retracted: A reviewer speaks out,” biologist Ben Ashby writes:

The problems of post-publication peer review have already been highlighted elsewhere, and it certainly isn’t rare for a paper to be retracted due to an honest mistake (although most retractions are due to misconduct). Moreover, one could argue that the mistakes in Kokko and Wong’s 2007 paper were sufficient to warrant a retraction as they significantly affected the conclusions. But by that logic, a large number of empirical studies should also be retracted due to incorrect statistical analyses or overreliance on fickle p-values, leading to irreproducible results.

OK, I have no problems so far, except to note that this is never gonna happen.

The part I don’t like is what comes next:

My concern is that the forced retraction of the original paper sends a bad message to the scientific community. Kokko [co-author of the original paper that got retracted, and also of the later paper that pointed out the error] has effectively been penalized for critiquing her own work, when in fact she should be applauded for her honesty.

No no no no no! Retraction is not a “penalty,” it is just a matter of correcting the scientific record.

Maybe there should be no such thing as retraction, or maybe we could ban the word “retraction” and simply offer “corrections.” That would be fine with me. The point is never to “expunge the record,” it’s about correcting the record so that later scholars don’t take a mistaken claim as being true, or proven.

But, to the extent there are retractions, or corrections, or whatever you want to call them: Sure, just do it. It’s not a penalty or a punishment. I published corrections for two of my papers because I found that they were in error. That’s what you do when you find a mistake.

I agree with Ashby when he writes:

These problems need to be highlighted at source (i.e. as corrections/erratum next to the original paper), with readers directed to the new paper for a fuller exploration of the corrected model.

I just don’t see how this differs from any other correction or retraction in the scientific literature. When there is an error, you want it to stay in the record, along with its correction.

Summary

Ashby writes:

In the case of Kokko and Wong (2007), does a retraction achieve anything that an Erratum or Technical Comment could not? In my opinion, no.

I agree and will go one step further. Does a retraction (in the sense of expunging the original material from the published record) ever achieve anything that an Erratum or Technical Comment could not? In my opinion, no.

P.S. Just to elaborate slightly: In general, I don’t imagine this correction procedure will be done by journals. There are just too many papers with serious errors. Even if only 10% of published papers have serious errors, that represents millions of possible corrections to be written, evaluated, and published. So I expect most of this will be done using external post-publication review.

But, when publications do want to correct the record, I think it makes sense to do so with corrections rather than retractions.

27 thoughts on “No Retractions, Only Corrections: A manifesto.

  1. >No no no no no! Retraction is not a “penalty,” it is just a matter of correcting the scientific record.

    When career/tenure decisions depend on the number of papers published by an individual, then retractions are a penalty. They aren’t a penalty in some ideal world perhaps, but they are when you are one of the few doing it, and your job depends on it.

    • +1

      To see retractions as not being a penalty at all ignores reality.

      OTOH, could one see them as a *deserved* penalty? In the absence of such things what disencentive do researchers have to not publish absolute crap or fib?

      • Rahul:

        It’s a cost, sure, but I don’t see it as a penalty. Just by analogy, suppose I dent your car and I have to pay $500 for the repairs. I’m paying the cost of my actions. But I don’t see the $500 as a penalty, it’s just a cost that someone has to pay.

  2. KD:

    Of course if you publish something that’s wrong, you shouldn’t get credit for publishing something that’s right. It’s not a matter of penalizing, it’s just accuracy.

    • Luis:

      Superseded is fine. But I think correction is also helpful.

      Consider two scenarios:

      1. Paper A reports an experiment. Paper B reports a more general experiment which allows the results of A to be interpreted in a new light. Paper B supersedes paper A.

      2. Paper A reports an experiment. Paper B, among other things, describes some mistakes in Paper A. Whether or not B supersedes A, we now know that A has an error. It should be corrected.

      • Yes, I understand that distinction (it may even be the case that paper A shows a fact on a smaller sample and paper B confirms it on a larger sample, then B supersedes A even though they point in the same direction).

        I would be fine with superseded being sometimes used as a polite lie. It’d still be an improvement on the status quo where authors will fight against a retraction.

    • John:

      For plagiarism, a note saying that his material was taken from whatever source. I guess there are some cases where so much is plagiarized that the material would have to be removed for legal reasons. Other than that, I’d keep the original paper but just label the source.

      In all these cases I guess it would be good to put some sort of watermark on the paper so that it’s clear to people who encounter it online, that there are corrections.

  3. I think there’s a distinction between:
    a) What it’s called
    b) And whether or not it is removed.
    Generally, I certainly prefer it never be removed, but always with an embedded description, and then some kind of watermarks (which might just be a a note that corrections, errata or replacements exist and where). Among other things, old copies propagate, but at least new ones wouldn’t without the watermarks.

    On the plagiarism/copyright issue, would adding a big diagonal “Plagiarism” in 120pt red type and a source to each page be enough to satisfy copyright holders? (I don’t know answer to that one. It advertises the originl and is miserable to read.)

    • John:

      Yes, exactly. I always thought “retractions” were fine, but reading this post by Ashby, I got the impression what when a paper is “retracted,” this involves the removal of the paper. I think it should stay but with corrections. Whether this is called a retraction or a correction doesn’t really concern me.

  4. What if the data was simply fabricated (e.g., the papers published by Diederik Stapel). Would the correction or technical note simply state that the data never existed?
    Of course, even retracted papers continue to get heavily cited in some instances because the journals bury the retraction so that no one becomes aware of the errors.

    • Mark:

      Yes, I’d give a note saying that there is no evidence the data ever existed. Still good to have the original paper there so people can see what the fuss was all about.

  5. @Andrew: The problem is penalties are the wrong incentive structure. People will be unwilling to look at their own results too closely for fear that it might have negative consequences to their career. There is a joke that goes around in experimental circles, if your results look good, don’t replicate. While this is a joke, I wouldn’t be surprised if a lot of people subconsciously did something like this.

    To me the right approach is not thru penalties, but by positive reinforcement of retractions. Those who retract need to get credit in some form for their work. Credit that helps their career and work-place requirements. This needs to be fleshed out. But, without such an incentive structure, we can talk all we want about how things should change. But, when one’s career depends on it, one finds a way to justify some serious violations sometimes. To berate such people as doing a disservice to science or to get on a high-horse and talk about morality to me simply avoids dealing with the reality of the situation.

    Are there cases where people have looked and questioned their own results. Surely. But, a few cases don’t make an argument. What we need is to think of the institution, and think about the correct incentive structure to encourage such behaviour. And, I’m afraid “punishment” is too deeply a religious concept for me to accept anyway as a sensible/reasonable solution.

    Note: I am not talking about fabricated data here. There surely needn’t be any positive incentive for retraction of such acts – to me, despite the high profile cases in the last decade, they are far and few between to worry about right now. There are simply bigger fish to fry. I’m talking about erroneous data (thru accidents or improper experiment design…).

    • KD:

      Incentives are great. But, just to be clear, I never suggested any penalties!

      You write, “I’m afraid ‘punishment’ is too deeply a religious concept for me to accept anyway as a sensible/reasonable solution.” So let me re-emphasized that I never suggested punishing anybody.

      Indeed, here were my exact words above, regarding the retractions or corrections that I recommend:

      It’s not a penalty or a punishment.

      So if you’re arguing against punishment, you’re not arguing with me. I’m on your side!

      In any case, this discussion is entirely theoretical as in most cases people simply refuse to admit they made a mistake. Even the himmicanes people and the air pollution in China people, to take two recent examples where the analyses were obviously flawed, have as far as I know never considered the possibility that they might have performed inappropriate analyses of their data.

      Heck, even Case and Deaton never acknowledged that they should’ve age-adjusted their death rates and separated them by sex. Even though everyone in demography knows to age-adjust death rates and separate by sex.

      There’s lots of people who really, really don’t like to admit they made a mistake. So I think we can forget about routine use of any correction or retraction process.

    • As a practical matter, how many non-robust results were ousted because the authors *themselves* published they were non-replicable?

      I always assumed it was other people who find the non-replicability of results.

  6. “Retraction”, rightly or wrongly, carries a strong tone of punishment and many negative consequences (e.g., in tenure review). As such, it should be reserved for cases of gross misconduct (fabrication, falsification, plagiarism). For analysis/design flaws, errors, mistakes, even if quite large (e.g., power posing), a “correction” or “update” appended to the original article (including appropriate changes to the substantive conclusions of the paper–editors must hold authors accountable for making these), is more appropriate because it corrects the record while also not creating (as large) a disincentive for authors to bring forth their own errors. This is especially the case for cases of researcher degrees of freedom–in many cases, authors probably didn’t realize at the time the problematic choices they made; punishment is not a good response to learning.

  7. Hi Andrew,
    I’ve only just come across your blog and found it really interesting. I think your views are similar to those expressed in a proposal that a group of us have been working on and posted to BioRxiv.

    The latest version is here:
    Amending Published Articles: Time To Rethink Retractions And Corrections?
    http://biorxiv.org/content/early/2017/03/24/118356#comment-3230389134

    but you might also like to see the comments on an earlier version too:
    http://biorxiv.org/content/early/2017/03/21/118356

    Please feel free to add your thoughts. We are considering all the feedback we’ve received and hope to revise in due course.

    Best wishes,
    Elizabeth Moylan

Leave a Reply to Andrew Cancel reply

Your email address will not be published. Required fields are marked *