Retractions, retractions: “left-wing enough to not care about truth if it confirms their social theories, right-wing enough to not care as long as they’re getting paid enough”

Two news items.

1. A couple people pointed me to the uncovering of another fraudulent Dutch psychology researcher—this time it was Dirk Smeesters, rather than Diederik Stapel. It’s hard to keep these guys straight—they all pretty much have the same names and do the same things.

Stapel and Smeesters also seem to live in the same postmodernist/business-school nexus: left-wing enough to not care about truth if it confirms their social theories, right-wing enough to not care as long as they’re getting paid enough.

In the comments to the Retraction Watch post, Richard Gill writes, “it looks to me [Gill] like Smeesters was subjected to medieval torture and confessed.” Medieval torture, huh? I haven’t seen Holy Grail in many years but I recall that’s pretty rough stuff, of the sort that even John Yoo might think twice about. I followed the links and didn’t see what the torture was, but I have to admit I didn’t even try to read the Dutch documents. On the upside, Gill follows up with “it seems like he did indeed have something serious to confess to, since all of his data is missing and no-one else ever saw any of it”—so maybe the medieval system worked in this case! Evilicious, indeed.

2. Thomas Basbøll pointed me to a recent argument by Fabio Rojas that retractions are good for science. But, as Thomas points out:

These retractions come after a great deal of criticism . . . Not only is the scholarship that is required to force a retraction difficult and time consuming to carry out, it often meets resistance from peers and editors. The retraction is the easy part. It is criticism that “makes science work” (in both senses).

My impression is that non-retraction is the norm, and retraction is considered exceptional. Just as we would applaud a person who rescues someone who has fallen on to the train tracks but would not criticize someone who does not perform the dangerous rescue, many academics see retractions as admirable but non-retractions as the usual behavior.

Thomas points to the work of Karl Weick, who has not been accused of faking his research results but has copied chunks of another’s work in his papers without attribution. Unlike the utterly discredited Stapel and Smeesters, Weick has followed the Wegman strategy of brazening it out and brushing aside all accusations of plagiarism. What Weick, Wegman, Stapel, and Smeesters all have in common, though, is that they are big shots within their fields but nobodies outside. Each has had some pre-scandal exposure (Weick has copied a story which has reached the ears of many famous people, Wegman testified before Congress, and Stapel and Smeesters have hit the headlines on occasion with their amusing research claims), but scholars outside their immediate fields generally don’t seem to have heard of these people.

Sometimes it seems that the people close to these offenders just don’t want to hear the bad news, and outsiders just don’t care. I agree, though, with Rojas that retractions are good. One challenge, though, is that uncovering the problem and forcing the retraction is a near-thankless job. That’s one reason I don’t mind if Uri Simonsohn is treated as some sort of hero or superstar for uncovering multiple cases of research fraud. Some people might feel there’s something unseemly about Simonsohn doing this (see several of the comments to the link at the very top of this post), just as some defenders of Karl Weick mocked Basbøll for going to the trouble of exposing a decades-old plagiarism.

OK, fine, but let’s talk incentives. If retractions are a good thing, and fraudsters and plagiarists are not generally going to retract on their own, then somebody’s going to have to do the hard work of discovering, exposing, and confronting scholarly misconduct. If these discoverers, exposers, and confronters are going to be attacked back by their targets (which would be natural enough) and they’re going to be attacked by the fraudsters’ friends and colleagues (also natural) and even have their work disparaged by outsiders who think they’re going too far, then, hey, they need some incentives in the other direction. So, yes, I think it’s fair enough for the Uri Simonsohns of the world to get a little fame and fortune in return for their admirable efforts.

23 thoughts on “Retractions, retractions: “left-wing enough to not care about truth if it confirms their social theories, right-wing enough to not care as long as they’re getting paid enough”

  1. Very interesting post. Just wanted to note that it actually often seems the reverse: right-wing enough to not care about truth if it confirms their social theories, left-wing enough to not care as long as they’re getting paid enough.

  2. I think most fields in the managerial sciences, including organization theory, would benefit greatly from establishing a tradition of publishing “merely” critical scholarship, i.e., scholarship that corrects errors in previously published scholarship. I don’t think there would be any loss (and great gain) if we published half as many empirical studies (of the kind that, say, Smeesters had an incentive to fake) and filled those pages (i.e., half of each issue of a journal) with straight correctives. I don’t mean forced retractions (i.e., notices) but the detailed results of the painstaking scholarship (or statistical analysis) that is required to find and correct errors. (Why is the Frank Fischer case only a PDF file attached to a CHE article?)

    Those errors will not always constitute fraud, of course. But in the current environment, the only (or at least the highly favored) way to challenge any already published theoretical claim is to conduct (or, I we now see, concoct) an empirical study. (I know my work constitutes an exception. But given the many journals that rejected my work before it was finally published, I’d confident in suggesting that I’m the exception that proves the rule.)

    By making critical articles a normal part of social science, there would be the standard academic incentive: you could improve your publication list by exposing error and fraud. You could even make a whole career out of that kind of activity. I’m sure there are people who wouldn’t approve of someone who built a career on pointing out the mistakes of others. But it might, as you rightly suggest Andrew, counter-balance the way fraudsters (and just plain sloppy scholars) are currently incentivized.

    • Thomas: Great idea. Perhaps there could even be a little work for philosophers/methodologists of science, given the right training.

      • Yes, I’ve been calling what I do “inframethodology”, an attempt to get “under” the methods that support research results and into the quality of the “craft”. Philosophers could definitely contribute (I’m a philosopher by training.) But here too the conventional approach pulls in the opposite direction. “Philosophers of” (physics, economics, etc.) often become promoters of the disciplines they are studying. Sometimes they become critics, but only at a quite general level. Exposing individual errors of analysis or scholarship is not normally part of what a philosopher does.

        Stanley Cavell does in one essay point out that Freud misread the ending of “The Sandman”, or more likely misremembered it and didn’t go back to check. That is, Freud simply gets the story wrong.

  3. 1. Not sure how you define immediate fields, but Weick is a big name in management, sociology, philosophy etc. And 2 documents with +10K citations on Google Scholar. That’s a pretty big name.

    2. I find it extra-ordinarily important to distinguish clearly between plagiarism and faking/messing with original data. The former is bad for the academic institution, the latter can cost lives or even worse: Business and money.

    3. Lichtentahler recently retracted one of his papers (open innovation guy). So far due to an error…

    • Anon:

      1. I communicated with various people I know about Weick, and none of my colleagues had heard of him. The guy’s a big shot in his field but he’s no Steven Pinker.

      2. I agree. These are different sorts of scientific misconduct. On the other hand, if you’re a Weick or a Wegman, faking original data is not an option on work in which you don’t collect data yourself. Plagiarism is one of the remaining ways you can cheat.

      • Ad 1. Sounds like poor sampling. Just look at citation patterns, across fields. I don’t know Pinker, btw. To add data, “steven pinker” has 1.4 million hits on google, while “Karl Weick” has 900.000. A difference, but not really between big shot and nobody + the former might include other Pinkers, while KW is arguable a far rarer name.

        • Oddly enough, *”Karl Weick”* (with quotes) gets 900,000 hits, while *Karl Weick* (without the quotes) gets only 300,000. I’d think the unquoted version would have more, but I guess this just means I know nothing about Google’s algorithm!

          Hmm, I think this is worth a blog entry . . .

        • Weird:
          Google gives me what I’d expect:
          331,000 without quotes
          134,000 with quotes

          Now, that is *without* being signed into to Google.

          If I sign in, the first number changes to 283,000.

  4. There is yet another update to the Wegman story.

    http://deepclimate.org/2012/07/13/wegman-and-said-leave-wiley-journal-and-said-disappears-from-gmu/

    “The saga of statistician turned climate science critic Edward Wegman and his protege Yasmin Said has taken yet another strange turn. The pair’s tenure as editors-in-chief at the Wiley journal they founded three years ago quietly came to an unceremonious end recently, while release of the hard-cover encyclopedia based on the journal also appears to have been delayed. Not only that, but it now seems that Yasmin Said’s stint as research assistant professor at George Mason University ended at the same time.”

  5. “A couple people pointed me to the uncovering of another fraudulent Dutch psychology researcher—this time it was Dirk Smeesters, rather than Diederik Stapel. It’s hard to keep these guys straight—they all pretty much have the same names and do the same things.”

    I wonder, I wonder what “DS” might be a common abbreviation for in Dutch!?!?

  6. Replication is overrated. How do things usually work? I have a story from early on in my career. About 30 years ago, I published a little piece, which I liked quite a bit – mildly important question both in theory and policy, good theory, neat results. Shortly thereafter a grad student and I, using a larger, overlapping data set, replicated my earlier analysis. We got nothing. We checked my earlier results, no mistakes; but the strong relationships I had found clearly disappeared when the data set was expanded, which is a pretty damning result. I encouraged my student to write a note to that effect, correcting my earlier article, which he did, but the editor summarily rejected it, noting the findings just weren’t interesting. In point of fact they weren’t. If our findings had been consistent with my earlier results, we would have published a second paper and, perhaps ultimately, a series of papers pursuing the implications of the initial findings. In other words, we would have replicated the analysis and eventually somebody might have noticed. At that point, showing we were in error would have been interesting. But that wasn’t the way things played out, and in the natural way of things, after a couple of citations, interest in the matter died. Ours first.

    The lesson I learned from that experience was to be a lot more careful, skeptical of my own work (and that of others as well). It is easy to make a mistake; more importantly, it’s easy to accept a result you like. Also, I deduced that from an external perspective the line between error (something we are all prone to) and fraud (which takes persistent effort) is a pretty fine one, especially when the researcher gets attached to a particular story, but important frauds are likely to be detected where scholars are required to make their data available to others. Important errors are probably harder to catch.

    One further point, Weick is a very well known scholar. I knew who he was long before I ever heard of Pinker. Pinker is a truly first-rate scholar; he is widely known as a public intellectual. The big difference is that one might read Pinker out of general interest, for enjoyment even, never Weick. If I asked my college classmates, at least the ones on the class listserv, I am sure that over half would recognize Pinker’s name. I’d get a blank for Weick. But if I were to ask my colleagues the same question, the percentages would be practically reversed.

    • Fred:

      In your case the gradual lack of interest in the work may not have been a problem. Indeed, some of my own work has ultimately led nowhere. But in a high-stakes area such as medical treatment or economic policy, I imagine that real harm could be done, either from unwarranted confidence in a mistaken claim, or conversely in a real finding whose acceptance is delayed out of fear that it is spurious. No statistical procedure can fully resolve this tradeoff, but fraudulent results and other versions of research misconduct (such as plagiarism and dishonest statistical analysis) can poison the well and reduce the ability of people to find signal among the noise of all the published studies.

      So I think that concerns about integrity are no joke, and I am bothered by people who don’t recognize and reward the effort that goes into studying this problem.

      • Fortunately, some of us don’t have to worry about tenure or citation-counting :-)
        As it turns out, there is yet another Wegman issue today, albeit not plagiarism.

        This is:
        1) running someone else’s code (which used unrealistic AR parameters and an amazing 1:100 cherry-pick, i.e., not exactly credible statistics)

        2) claiming to have validated that, i.e., an independent replication (as opposed to just running code and getting same answers)

        3) when asked for the code by a US Congressman (since it was in a high-profile report to Congress), promised a website (which never happened), but claimed it would take a while as it required Navy approval. FOIA: Well, no, this was just an excuse, since the Navy never had this part of the code, and the rest was releasable.

  7. HERE’S A PAPER THAT SHOULD BE RETRACTED. Reliable nutrition information is critical in the fight against obesity and diabetes. In Australia, the contribution of excess sugar consumption to obesity has been exonerated by high-profile but over-confident scientists with strong links to the sugar industry and other sugar sellers.

    No surprise I guess, but what’s interesting is that this deeply flawed paper with its spectacularly false conclusion was published in a supposedly peer-reviewed science journal. Dr Rosemary Stanton of the University of NSW and Professor Boyd Swinburn of Deakin University both have agreed publicly that the authors’ conclusion belies the readily available facts.

    I’m arguing far and wide for the shoddy paper’s retraction by the authors, the journal Nutrients and/or the University of Sydney. It’s all documented at http://www.australianparadox.com/ and http://www.smh.com.au/national/health/research-causes-stir-over-sugars-role-in-obesity-20120330-1w3e5.html#ixzz20FXohd4R .

    Nothing has happened since March except that the authors – Professor Jennie Brand-Miller and Dr Alan Barclay – have pretended their paper is fine. It isn’t. And it’s simply unreasonable to allow the false conclusion – “an inverse relationship” between sugar consumption and obesity, the Australian Paradox! – to sit uncorrected in a journal, misinforming scientists across the world via the Internet many months after the real facts have become clear.

    Because of the unreasonable delay in correcting the scientific record, one of the questions I’m asking is when does an inadvertent series of major errors deliberately left uncorrected become an academic and scientific hoax? Any thoughts, anyone?

  8. Reading the article on responses on Retraction watch, it looks like the fraud was detected through FDR or false discovery rate calculations.
    However, I have also heard that Simonsohn uses p-hacking, which is “the idea that if researchers are engaging in questionable analysis practices, then they should have a disproportionate number of findings at or close to the p < .05 threshold for statistical significance"
    Other responses point to discrepancies in degrees of freedom which indicates some subjects were dropped.

    Could you please comment on to the extent to which these are different, or to what extent they could overlap or are the same?

  9. Pingback: Qualitative Coding as Ritual: A Review of Biernacki’s “Reinventing Evidence in Social Inquiry” « A (Budding) Sociologist's Commonplace Book

  10. Pingback: Uri Simonsohn is speaking at Columbia tomorrow (Mon) « Statistical Modeling, Causal Inference, and Social Science

Comments are closed.