The retraction paradox: Once you retract, you implicitly have to defend all the many things you haven’t yet retracted

Mark Palko points to this news article by Beth Skwarecki on Goop, “the Gwyneth Paltrow pseudoscience empire.” Here’s Skwarecki:

When Goop publishes something weird or, worse, harmful, I often find myself wondering what are they thinking? Recently, on Jimmy Kimmel, Gwyneth laughed at some of the newsletter’s weirder recommendations and said “I don’t know what the fuck we talk about.” . . .

I [Skwarecki] . . . end up speaking with editorial director Nandita Khanna. “You publish a lot of things that are outside of the mainstream. What are your criteria for determining that something is safe and ethical to recommend?”

Khanna starts by pointing out that they include a disclaimer at the bottom of health articles. This is true. It reads:

The views expressed in this article intend to highlight alternative studies and induce conversation. They are the views of the author and do not necessarily represent the views of goop, and are for informational purposes only, even if and to the extent that this article features the advice of physicians and medical practitioners. This article is not, nor is it intended to be, a substitute for professional medical advice, diagnosis, or treatment, and should never be relied upon for specific medical advice.

. . . I ask: “What responsibility do you believe you have to your readers?” Here at Lifehacker, I recently killed a post I was excited about—a trick for stopping kids from unbuckling and escaping from their car seat—after a car seat expert nixed it. I feel like if I’m providing information people might act on, I have a responsibility to make sure that information is reasonably accurate and that people won’t hurt themselves (or their children) if they take me at my word.

Goop’s editors don’t see it that way. “Our responsibility is to ask questions, to start the conversation,” Khanna says.

OK, so far, not so bad. Goop’s basically an entertainment site. It’s goal is to be thought-provoking. They’re not heavy on the quality control, but they make this clear, and readers can take Goop’s articles with that in mind.

From this perspective, to criticize Goop for peddling pseudoscience would like criticizing David Sedaris for embellishing his Santaland story. It’s beside the point. We can’t get mad at Goop like we can get mad at, say, Dr. Oz, who’s using his medical degree and Columbia University affiliation to push questionable products.

But then Skwarecki continues:

I turn the conversation to Goop’s infamous jade eggs. They are for sale that day in the pharmacy shop, and I got to hold one in my hand. It was smaller than I expected, not the size of a chicken egg but more like a grape tomato. Both the jade and rose quartz eggs have a hole drilled through the smaller end, and at first I imagined a Goop acolyte taking the egg out of her vagina, rinsing it off, and hanging it around her neck. I learned later that the hole is the answer to the question in the jar: you can attach dental floss to give it a removal string, like a tampon.

The idea of the jade egg, or its prettier rose quartz companion, is to “cultivate sexual energy, increase orgasm, balance the cycle, stimulate key reflexology around vaginal walls.” The grain of truth here is that using a small weight for vaginal exercises can help strengthen the muscles in that area. You can do this without a weight, too.

But Jen Gunter, a practicing gynecologist who is one of Gwyneth’s most vocal critics, has explained that jade eggs are a terrible idea. Stones can be porous enough to grow bacteria, and she says the instructions for using the egg are incorrect and could harm people. For example, a Goop article suggests walking around with the egg inside of you. Gunter counters that overworking your vaginal muscles this way can result in pelvic pain.

The Goop editors remember the jade egg backlash, and they are unfazed. “Did you read the letter from Layla?” Khanna asks. Layla Martin, who sells jade eggs and a seven-week course on how to use them, wrote a 2,000-word “letter to the editors” defending the eggs. Goop published it in their newsletter, and underneath it, their disclaimer, and underneath that, a link to their shop.

Whoa! That doesn’t sound like healthy living.

The punch line:

Khanna says they “never considered backing down.” She points out, as if it were a defense, that the eggs were very popular and sold out right away. I ask her: Has there ever been a health article in Goop that you thought afterward, maybe we shouldn’t have run that?

No, she says, never.

Interesting story. It reminds me of Freakonomics. I always wondered why they never retracted some of their more embarrassing mistakes, such as their endorsement of Satoshi Kanazawa’s silly claims about beauty and sex ratio, or their breathless coverage of Daryl Bem’s ESP paper, or their book chapter on climate change. Why not retract the errors that experts point out to you? My best guess was that they didn’t want to start retracting even their biggest goof-ups, because once you start retracting, you’re implicitly endorsing all the things you didn’t retract. Paradoxically, if you don’t really believe the things you’re writing, you might be better off not retracting anything.

the Freakonomics team and the Goop team are in the same place, which is that they believe they are fundamentally doing good by spreading the principles (healthy living for Goop, economics for Freakonomics) and that the details don’t really matter. They know they’re the good guys and they don’t want to hear otherwise.

P.S. Skwarecki’s article is on Lifehacker, a site formerly connected with Gawker. Now that they’re being mean to a business venture, I wonder if Peter Thiel will try to sue them to death. I hope not.

P.P.S. The comment thread on Palko’s post continues in some interesting directions, including a discussion of some subset of eminent journalists and scientists who seem to care more about the well-being of their professional colleagues than anything else.

14 thoughts on “The retraction paradox: Once you retract, you implicitly have to defend all the many things you haven’t yet retracted

    • evidence can’t make you update away from certainty.

      Sure it can, it just means altering some underlying assumption (which by definition cannot be certain). Eg I am quite certain that, given certain assumptions, 2 + 2 = 4. You could possibly convince me that one of those assumptions does not apply to our universe though.

      Similarly I am quite certain that NHST is logically invalid, however perhaps the system of logic I am using is based on some incorrect premise.

      • Anoneuoid,
        I think there may be some miscommunication here. I would guess that DM is referring to people who accept underlying assumptions as certain.

        • I assumed he was talking about updating beliefs using Bayes’ rule. For example say we are estimating some parameter of a model. His claim would amount to putting prior probability of 1 on one value and zero on all others. I am saying evidence can make us change the likelihood/etc to get around this (ie the parameter in question isn’t even relevant any longer).

  1. Two thoughts in opposite directions.

    1. If you acknowledge that there is uncertainty in research findings and that findings can turn out to be wrong through no fault of the researcher, then yes, it makes sense for honest researchers to retract incorrect claims.

    2. But, when you add pesky real-world concerns, even an honest researcher may not want to retract incorrect claims.

    a. Charlatans will retract a claim only if forced to do so by massive pressure. They can often successfully cover up their mistakes. An honest researcher will, therefore, look more dishonest than all but the most reckless charlatans.

    b. The ability of outsiders to understand the issues is close to nil. A single retraction is therefore given outsized weight.

    c. Hostile parties will use the retraction to discredit everything you have done. Tagging Weick as a plagiarist is an example.
    This is not ridiculous. In fact, it is a basic legal principal in evaluating credibility: “false in one, false in all”.

    The concept of asymettric information is relevant to a surprisingly wide range of phenomena.

    • The information is symetrically assymetric.

      1. Honesty requires careful evaluation of sources of uncertainty and perforce acknoweldgement of it. (Corollary: honest criticism does not demand certainty in support of a proposition.)

      2 a’ Failure to react appropriately to criticism short of massive pressure makes you look like a charlatan. Honest researchers take responsibility for their past work. (That’s difficult when you are dead, I suppose. Heirs to the research have some basic responsibility, but that’s a tough one.)

      b’ The ability of outsiders to understand the issue of stonewalling is close to 100%, so it gets the weight it deserves.

      c’ Hostile parties will extrapolate the failure to respond. But here they are on more solid ground, because this failure is evidence of moral character, not merely faulty reasoning or mayb ignorance. (Peer review does have a responsibility to catch gross ignorance or careless reasoning.)

  2. Prof. German: your comparison of goop and freakonomics is unfair. There is a difference between weak evidence and no evidence. You say the Kanazawa, Bem,etc stories are silly and I agree, but they *do* present evidence in refereed journals. Yes yes it’s no guarantee but it’s not the same as making empirically vacuous claims. Moreover the freakeconomics guys are not selling the subject of their stories.

    • Jack:

      The Kanazawa and Bem papers are no better than reports of random numbers coupled with empty, open-ended theory. They were published in refereed journals and that’s too bad. Referees can make mistakes, and in those cases they did.

    • Another difference seems to be that freaknomics makes pre-publication review of the content, while goop doesn’t (based on your description, since I’ve never heard of goop before). To put it another way, if you think they have the same credibility, you give the same probability for a claim in goop be right as a claim on freaknomics. But if so, you would also accept a bet based on such odds.

  3. Here’s an article on “Retracted Science and the Retraction Index” https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3187237/pdf/zii3855.pdf

    “Articles may be retracted when their findings are no longer considered trustworthy due to scientific misconduct or error, they plagiarize previously published work, or they are found to violate ethical guidelines. Using a novel measure that we call the “retraction index,” we found that the frequency of retraction varies among journals and shows a strong correlation with the journal impact factor. Although retractions are relatively rare, the retraction process is essential for correcting the literature and maintaining trust in the scientific process. We defined a “retraction index” for each journal as the number of retractions in the time interval from 2001 to 2010, multiplied by 1,000, and divided by the number of published articles with abstracts. A plot of the journal retraction index versus the impact factor revealed a surprisingly robust correlation between the journal retraction index and its impact factor (P 0.0001 by Spearman rank correlation) (Fig. 1). Although correlation does not imply causality, this preliminary investigation suggests that the probability that an article published in a higher-impact journal will be retracted is higher than that for an article published in a lower impact journal.”

Leave a Reply to Martha (Smith) Cancel reply

Your email address will not be published. Required fields are marked *