Mmm, statistical significance . . . Evilicious!

Just in case you didn’t check Retraction Watch yet today, Carolyn Johnson reports:

The committee painstakingly reconstructed the process of data analysis and determined that Hauser had changed values, causing the result to be statistically significant, an important criterion showing that findings are probably not due to chance.

As the man said:

His resignation is a serious loss for Harvard, and given the nature of the attack on him, for science generally.

As a statistician, I don’t mind if someone is attacked because of cheating with data.

Johnson concludes her news article in a pleasantly balanced way:

The committee said it carefully considered Hauser’s allegation that people in his laboratory conspired against him, due to academic rivalry and disgruntlement, but did not find evidence to support the idea.

The committee also acknowledged that many of Hauser’s overall findings about the cognitive abilities of animals may stand. His results that showed that animals may have some of the same cognitive abilities as people have been important for the field. But science depends on the data.

“Skepticism above all toward the veracity of one’s own hypotheses is, of course, an essential virtue for scientists,” the committee wrote, “and one that must be modeled for the benefit of trainees.”

P.S. Given the title of this post, I should probably clarify that I do not blame statistical significance for Hauser’s offenses. Any statistical procedure of any sort will be destroyed if you start monkeying around with the data.

P.P.S. In all seriousness, I wonder if Hauser feels more relaxed now that he’s been caught and these conclusions are out in the open. He can spend more time doing the theorizing he loves, rather than the experimental work that freaked him out so much. Shifting to this new career couldn’t have been so easy for him, but now that it’s happened, maybe he’s reached more of a state of calm.

The next step, I hope, is confession and apologies to all the people he accused, all the people who wasted so much time on this case (including the people on that committee), his former graduate students (also, maybe some reparations to the person who resigned from his lab after Hauser said he was “a bit pissed” to have his work questioned), and, not least, to Hauser’s friends, the people who stood by him and defended him through all the denials. After that, maybe he can move on with his life.

46 thoughts on “Mmm, statistical significance . . . Evilicious!

  1. “As a statistician, I don’t mind if someone is attacked because of cheating with data.”

    Yes, BUT: he cheated with the data in good faith regarding publication and his career. I guess you feel bad NOW!

  2. Andrew,

    regarding your P.P.S., this reminded me that Diederik Stapel has been going through an extended tour of apologies in recent times. Any thoughts on this, you haven’t blogged about him for some time now…

    Or, to put it differently (Stapel may be too extreme an example, though): How should academics recover from such self-inflicted damage (recover in terms of getting back into research)? How should journals react? How should colleagues react? Would you publish with Hauser, and if yes, under which conditions (for the sake of the argument, let’s say that you have a common research interest)?

    • Martin:

      It’s not up to me in any sense, but my impression is that Hauser can’t handle the ambiguities that arise in empirical research, and he should probably stick with theorizing. Or maybe he’s really good at working with at-risk youth, I have no idea.

      Would I publish with Hauser? No. (I’m assuming we’re talking realistic conditions here, of course I’d publish with him if someone were holding a gun to my head etc.)

  3. Lets see if we can improve this without getting too wordy:
    “statistically significant, an important criterion showing that findings are probably not due to chance”

    “statistically significant. As commonly (but not originally supposed to be) used, this is a criterion determined by social convention important for determining if researchers are able to publish their results. Meeting this criterion shows that the difference between findings and the opposite of a researcher’s hypothesis would be surprising if due to chance alone.”

    I think it at least suggests the important questions to be asked.

  4. You may have read these stories lately:

    1. Jared Remy pleads guilty to murdering Jennifer Martel, the mother of their child. (He was literally pulled off her by neighbors.) He says in his sentencing statement that it’s all his fault … and then blames her for, as he says, threatening him with their child.

    2. A 6 year old in South Carolina pulls the trigger on an AK-47 left on a table at a family gathering. Kills his grandfather. The kid’s uncle had loaded the weapon to show it off to his father, the man killed. The now dead grandfather put the gun down and they walked away. Reaction: it’s no one’s fault. It’s a tragedy.

    We seem to hard wired to deflect blame. The first example I’m aware of is Adam and Eve, the first creation story (though the second by age): did you eat the fruit? Well, she said to. Did you eat the fruit? Well, he told me to. I gather there was a tree of knowing things but not a tree of wisdom in the Garden.

    Here’s a 3rd:

    Rudolf Höss wrote an interesting memoir while a prisoner of the French. They had him under psychiatric examination and the memoir was a product of this. He was the guy who ran Auschwitz for about 3 years, as I remember, and was eventually hung. The memoir describes the tremendous difficulty his men had killing people who looked just like them, who in many case resembled their own mothers or sisters or brothers or their own children. They needed counseling to be able to continue to murder. The point is they knew they were killing people. Höss accepts that as absolute fact: he is guilty of killing people. But he doesn’t claim “I was just following orders” but instead says it was the fault of the people they were murdering that they existed and had to be killed.

    We as a species are capable of that degree of dislocation of blame. So don’t expect Hauser to emulate Jacob and confess and offer repentance.

    • Jonathan:

      Good point. But I’ve also read about the relaxation that is felt after confession. I’m guessing that Hauser’s life as lab director was highly stressful, with a lot of pressure to keep coming up with experimental findings (indeed, I don’t blame statistical significance but I do blame the expectation, implicitly encouraged I think by statistics textbooks, that if you run experiments you will routinely get interesting and statistically valid results), whereas maybe his current life is more pleasant. “I resigned to spend more time with my family” is a cliche but maybe there’s some truth to it.

  5. In some sense, it’s too bad that the investigation committee names are redacted (as they usually are).

    They are owed congratulations for what has to be one of the most thankless tasks in academe, as the recognition/reward ratio is probably even lower than for doing paper reviews.

    I’m reminded of the hassles of the folks who did the Ward Churchill case.

  6. I wonder why, in this and similar cases, it is rarely talked about the monetary aspect (that’s my impression at least). After all, not only did he probably waste a lot of public funds, at the same time he took the funds away from other researchers who might have put them to a more reasonable use. This is a twofold damage to the public. I also wonder why people like Hauser cannot be held personally accountable for this damage by having them pay back the money they wasted? Can’t institutions like NIH sue somebody like Hauser? And: why does stuff like this not become subject of law enforcement? After all, he acquired money under false pretense for the sole purpose of his own symbolic and material enrichment.

    • If you want to bring in law enforcement, why not send cops after someone like Tol who makes a rash of data entry errors? Is that not culpable negligence? I bet the net monetary impact of Tol’s mistake is higher than Hauser’s.

  7. It’s ironic that Tol can blame gremlins & get away with it but not Hauser, eh?

    We as a system focus on the easiest to prove offenses not the highest damage ones. That’s what annoys me each time we gloat over a fired plagiarizer or debarred data faker. The big damage lies elsewhere.

    The more insidious forms of lying using data, outright negligence etc. get away scot free. Andrew might critique Tol & other egregious characters endlessly, but in practice there’s no penalty in the system for them.

  8. My impression one would have to be an idiot to fabricate data in political science since it is so simple to do an analysis which gives the results one wants. The same is also true of psychology. Hauser is an idiot.

    • Numeric:

      I have no idea if Hauser is an idiot but he succeeded for quite awhile before he finally got caught and kicked out. He was flying high before the crash. And, indeed, if some of his colleagues are to be believed (and maybe they are right), many of his claims were both innovative and correct. So it’s possible that he’s an excellent theorist with deep understanding of psychology who just does not have a good understanding of variation and statistics. He could be a true believer in what Kahneman and Tversky called “the law of small numbers,” in which any underlying pattern is supposed to appear in any sample, no matter how small. So when he saw monkey behavior that didn’t fit the model, he cheated—but, in his mind, this cheating could’ve been done for the larger goals of science.

      I don’t agree with such behavior (of course) but it wouldn’t make him an idiot, it would just make him ignorant of a particularly tricky aspect of reality—that aspect that we call “statistics.” Also it seems that he was telling untruths, but lots of people do that. Again, if he felt he had a directly line to God, or to Nature, then maybe he felt that these were small untruths in service to a larger Truth.

  9. Pingback: O’s digest, arretrati – Ocasapiens - Blog - Repubblica.it

  10. Based on the evidence, or lack of, an investigation into Harvard is
    called for as a matter of urgency (“Harvard report shines light on ex
    researcher’s misconduct”, Boston Globe, May 30 2014)

    http://www.bostonglobe.com/metro/2014/0 … story.html

    The inconsistencies clearly evident in the Harvard Report raise very
    serious questions regarding the motives of those involved in the
    ‘Investigation’ of Dr Marc Hauser.

    For example:

    1(a) “Hauser then wrote an e-mail suggesting the entire experiment
    needed to
    be recoded from scratch. “Well, at this point I give up. There have
    been so many errors, I don’t know what to say. . . . I have never seen
    so many errors, and this is really disappointing,” he wrote.
    In defending himself during the investigation, Hauser quoted from that
    e-mail, suggesting it was evidence that he was not trying to alter data.
    The committee disagreed.
    “These may not be the words of someone trying to alter data, but they
    COULD certainly be the words of someone who had previously altered
    data…”

    1(b) “COULD” ??!!

    2(a) ‘Later that day, the person resigned from the lab. “It has been
    increasingly clear for a long time now that my interests have been
    diverging sharply from what the lab does, and it seems like an
    increasingly inappropriate and uncomfortable place for me,” the person
    wrote…’

    2(b) Question : Who was that “person”?

    Answer : “Much has been redacted from the report, including the
    identities of those who did the painstaking investigation and those who
    brought the problems to light”.

    I am reminded of two passages:

    1. Matthew 7 v 5

    2. “Many people presumably know that they have done something wrong
    based on reactions by others, but don’t admit to the wrongdoing or take
    responsibility. Some of these people are excessively narcissistic, a
    disorder that can bleed into the presidency…President George W. Bush
    failed to admit to the public that he went to war with Iraq for reasons
    other than the one concerning weapons of mass destruction…” ~ Marc
    Hauser (Source : ‘Moral Minds – How Nature Designed Our Universal Sense
    Of Right And Wrong”, Ecco 2006 – Page 155).

    An investigation into Harvard – and beyond – should take place
    immediately.

  11. Viking-Penguin had agreed to publish “Evilicious”, but pulled out at the last moment – leaving Hauser no choice but to get it published by Amazon, Kindle et al.

    I am growing in the conviction – with growing evidence – Viking-Penguin’s decision not to publish Hauser’s book was a very serious professional mistake – as well as an unforgivable breach of promise.

    Viking-Penguin, with such a history of publishing courage and integrity, clearly caved in to pressure from un-named legal and political forces – just like Harvard.

    “Evilicious” was the sequel to “Moral Minds” (Ecco HarperCollins 2006). In both books, Hauser has made a critical contribution within a ‘cutting-edge’ research field.

    It is deeply saddening, and disturbing, to know there is unlikely to be an admission of wrong-doing either by Viking-Penguin, or by Harvard.

    • Richard:

      It sounds like Hauser’s strength was in theorizing but he had trouble with empirical work. That’s fine—theorists have a lot to contribute to the world. Maybe now that he’s free of the burden of trying to interpret experimental data, Hauser can bloom in his theoretical work, even perhaps suggesting experiments that others can do.

    • “Unforgivable” breach of promise? Unforgivable for who? Viking-Penguin is a business, and one can’t expect a business to take actions that have a foreseeable adverse effect on the bottom line when a better alternative exists. In the capitalist system, it’s all in the game.

        • I wonder Corey, if there was a contract signed between Hauser and Viking, that you would be quite so dismissive?!

        • Richard, I am confused why you’re questioning Viking-Penguin. I assume in all their contracts they have the right not to publish the submitted work (and publishers often don’t, there are more contracts out there than books, often with the authors not submitting). Are you suggesting this right be taken away from them for this case, or are you just saying we as researchers should be against their decision? Fortunately for Hauser there are other alternatives.

        • Dan, I have it on good authority Viking-Penguin had an ‘understanding’ with Hauser they would definitely publish “Evilicious” – the sequel to “Moral Minds” (Ecco-HarperCollins).

          That ‘understanding’, as far as I know, held throughout the controversy – until they unexpectedly pulled out in 2012 without explanation.

          Of course a publisher has a right not to publish – but do they have the right to make firm promises of publication to a renowned writer/researcher (thus preventing other publishers from offering to publish), and then ‘pulling the carpet from under them’ at the last moment? I think not.

          I have the two books in front of me : “Moral Minds” (Ecco-HarperCollins) and its sequel “Evilicious” (Amazon). One will grace my library – the other will not.

        • Okay, that is a different interpretation of what the law says he has the right to do than how I would interpreted it, but that’s fine.

        • An ‘understanding’ is not a ‘contract.’ If I were publisher, and had agreed to publish a scientist’s work, I’d expect there’d be a mutual ‘understanding’ that the scientist was not engaged in fraud. If fraud came to light, I might decide to withdraw the agreement immediately, or I might give them the benefit of the doubt until more information came to light, for or against. Perhaps Viking-Penguin was hoping some sort of exoneration was coming, and when it didn’t after sufficiently long time, they gave up.

          As to your rhetorical question, “… do they have the right to make firm promises…,” yes, they do. If there were a contract, there would (most likely) be penalties spelled out for both sides in terms of what could and could not be used to break the contract, and Hauser might have standing to seek redress. As it was, there was almost surely an implicit understanding that Hauser was not practicing scientific fraud (at least on the part of Viking-Penguin). Which he almost certainly was, based on the redacted report.

        • As Corey said above [June 9 – 5:31pm] : “In the capitalist system, it’s all in the game”

          Well, I don’t know what “game” Viking were playing, but it certainly wasn’t a commercial game. Publishers publish to maximize profits under a capitalist system (especially US-styled capitalism). Nobody’s going to convince me that Viking ‘pulled the rug’ because it threatened their profits – and certainly not for ethical/moral reasons. If the latter, not many books would get published. “Evilicious” woud have been a money-spinner for Viking – the more controversy the better, as it increases sales – thus maximizes profits.

          No, Viking pulled out of the deal for a reason wholly different from commercial or moral considerations. To discover that reason requires courageous investigative journalism. That is in short supply – and certainly not evident at the Boston Globe.

        • Richard:

          I think it is an appropriate moral decision for a publisher to decide not to publish a science book written by a scientist who’s conducted research fraud. You might disagree with the consensus on this—maybe Hauser didn’t really misrepresent the data as the report said—but, given all the information we have, it’s certainly reasonable for the publisher to conclude that Hauser is untrustworthy, in which case it seems very reasonable for them to not want to publish his book.

        • Thanks Andrew.

          “given all the information we have, it’s certainly reasonable for the publisher to conclude that Hauser is untrustworthy”

          My point is that “all the information we have” is very little – certainly not enough information to pass judgement in the way so many are doing.

          The little information we have is based on an individual(s) whose name(s) have been redacted from a Harvard Report, after which Harvard did not – repeat not – fire him.

          There’s something that smells more than fishy here, and the source of that smell needs to be traced – or else all researchers, and their methods, are put at risk.

        • Of course I would. Incurring the civil wrong of breach of contract and giving Hauser legal cause of action to sue for redress is a business decision. Corporations aren’t people, my friend.

  12. That’s as maybe Andrew, Hauser might well have found more enriching and fulfilling work by his ‘fall from grace’, but does that give good enough reason for Harvard, ORI, Viking et al to walk away without blemish or blame? Methinks not. From where I’m sitting, this looks a clear case of a well-respected Professor being ‘stitched-up from on high and hung out to dry’ – while the likes of Harvard watched and did nothing – or not enough. Sadly, this targeted character assassination of certain academics – in the US and elsewhere – is nothing new.

    • Is there evidence that Hauser was targeted? I haven’t seen any aired, only accusations that he was. Even if he were targeted, he seems to have given his enemies plenty of rope to hang him with.

      • Ray, for facts and evidence we have to look beyond the Boston Globe – and a redacted Harvard Report. Sweeping generalisations are not facts or evidence.

      • Have we evidence that Hauser was targeted?

        We have evidence that Hauser’s Lab was ‘raided’ while he was in Australia – and everything was taken relating to his work…emails, hard-drives, day-to-day notes, unpublished materialetc etc.

        Selected details of a Harvard Report was then ‘leaked’ to the Boston Globe – and now we have that Report in full – with the names of the Committee members, and individual(s) involved, redacted.

        Because of that redaction there is no way of knowing if the Committee (un-named) – basing their report on an individual(s) (un-named) – carried out an investigation which was “painstakingly reconstructed” (Boston Globe)

        “The committee painstakingly reconstructed the process of data analysis and determined that Hauser had changed values”

        For all we know, the Committee could be guilty of the very thing for which they are accusing Hauser.

  13. This might be an ‘inconvenient truth’ to some, but it should not be forgotten Harvard never fired Hauser. They had every intention of allowing him to teach and carry on his research – and Hauser had every intention to return – but something happened…

    Can we honestly put our hand on our heart and say the punishment fits the crime in this case? You might. I can’t. Hauser wasn’t the only one at fault here. At least he admitted to his wrong-doing.

    • “Can we honestly put our hand on our heart and say the punishment fits the crime in this case? You might. I can’t. Hauser wasn’t the only one at fault here. At least he admitted to his wrong-doing.”

      Could not disagree more with this statement. In so many ways. Also, he didn’t admit his wrongdoing (unlike others in related fields who have been caught recently).

  14. Here is Harvard’s redacted full report:

    http://cache.boston.com/news/pdfs/harvardreport.pdf

    To say the redacted material (of name(s), reports, statutes) is “aggressive” is a serious understatement – which does raise an even more serious question as to who was responsible for the redactions.

    The Globe’s Carolyn Johnson has said “I can’t speculate about the redacted names”, which suggests she and the Globe – and Hauser & Harvard (& its students) – are legally bound not to make any mention of names…or else.

    Presumably, this legal matter involves the legal team of Alan Dershowitz at the Harvard Law School, who were heavily involved at the start of the controversy?

  15. This might not be entirely irrelevant – “Decadence” by CEM Joad (1948 UK/1949 US), an English philosopher whose ‘fall from grace’ in 1948 was rapid after the media-hyped train ticket ‘scandal’ – sacked by the BBC, but not by University of London’s Birkbeck – the philosophy department which he established in 1930 until his death 23 years later, aged 61:

    http://www.shuv.org/j_biblio.html

    Joad, C.E.M. Decadence: A Philosophical Inquiry (New York: Philosophical Library, [1949]). The late C.E.M. Joad (1891-1953) was a renowned philosopher on the faculty of the U. London (1930-1953). He published many works, including studies in ethics, theology and religion. In this study Dr. Joad defines decadence as “the arrogance of man getting above himself and thinking he is lord of the universe” (pp. 14-15). He concludes that a decadent society is caused by the loss of belief in a higher spiritual reality: “the conviction that what is good and beautiful has its origin in and derives its authority from some other plane of reason… [which] invests our own world with a significance it had otherwise lacked.” This conviction, Joad said, “forms part of the Christian tradition of Western Europe,” but is threatened by factors which tend to preempt it such as “material prosperity and technical advance” (pp. 12-13). Characteristics of a decadent society include: “luxury, scepticism, weariness and superstition… as well as a preoccupation with the self and its experiences” (p. 117).

    “A non-decadent community” Joad says, “is… one which is conscious of the spiritual order of the universe, more particularly as it manifests itself in values. Inhabitants of this order are God and the values in which God expresses and manifests Himself, namely, truth, goodness, and beauty…In decadent societies [this spiritual order] is lost sight of” (p. 281). Joad labels the materialistic cultural drift “insectification,” and states, anticipating Thomas Cahill, and perhaps supporting Niebuhr’s category: “Christ above culture,”: “Granted the theistic hypothesis we may, I think… suppose that some human beings will be proof against the general process of ‘insectification’ and will retain the standards and values of civilized men, or will retain at least their memory much as the saints preserved the Christian faith in pagan lands and the monks some remnants of classical culture during the Dark Ages… By withdrawing, a man might…hope to keep alive some remnant of the culture of an earlier and more civilized time” (pp. 399; 401).

  16. Pingback: Marc Hauser: Victim of statistics? « Statistical Modeling, Causal Inference, and Social Science

  17. Pingback: The flashy crooks get the headlines, but the bigger problem is everyday routine bad science done by non-crooks « Statistical Modeling, Causal Inference, and Social Science

  18. Pingback: When does a misunderstanding reach the point where it is recognized to be flat-out ridiculous? « Statistical Modeling, Causal Inference, and Social Science

Leave a Reply to BadYodeler Cancel reply

Your email address will not be published. Required fields are marked *