Blaming scientific fraud on the Kuhnians

I wouldn’t go that far, but I’ll send along this article by Ahti-Veikko Pietarinen that was sent to me by Lee Sechrest. Those of you who like this sort of thing might like this sort of thing. I neither endorse nor anti-endorse. Or, I should say, I am in sympathy with the author’s general attitude but I have not looked at any of the details.

39 thoughts on “Blaming scientific fraud on the Kuhnians

  1. Well I got five pages into it hoping to see Kuhn taken down a notch, but blaming Kuhn and Quine for Stapel is complete crap. If they’re responsible for Stapel, then what evil couldn’t you blame them for? I’m going to write a paper on how Kuhn’s to blame for bunions and people who wear socks with sandals. “Redonculus” as the kids say.

    • Oh, and the real reason fraud is accruing is because the pressure on researchers to produce is very high and increasing, while at the same time there’s very little progress being made. Neither of these looks to be changing anytime soon. I confidently predict there’ll be more knuckleheads who crack under the pressure and start scamming the system somehow.

      • Aristotle talked about how forms of government become perverse. In this sense, I think Stapel and the general mindset that feeds “looking for BIG results and BIG results only” is perverse Kuhnianism. Stapel is the tip of the iceberg but the pressure for big results is exactly what you’re indicting. I remember taking philosophy of science as a grad student way back when. Despite the professor being very careful to indicate that normal science was, well, normal for a reason, there was always an unspoken dinging of normal science, which is where progress is small and incremental. It took me quite a while before I managed to fix the mindset of a naive 22 year old.

        • Jay:

          This sort of thing also arises with the researchers who think they can discover enduring truths by asking a few questions to 100 Mechanical Turk participants. On one hand, the purported statistical significance of their results are justified on the basis of the results being in line with existing theory (thus, normal science at its normalest); on the other hand, the publication of such results is justified on the basis of it being a (small) breakthrough.

        • It’s a definite tension. Obviously we as a society would like big gains—and certainly university press offices do!—but without a good bit of normal science breakthroughs aren’t possible. It’s tricky because anyone who’s been a scholar for a while can identify a plodding researcher who’s just making small shifts unlikely to go much of anywhere.

    • As I sit here I am wearing socks and sandals and I have read Kuhn several times. This is supporting evidence, which you are free to use.

      But aside from possible fashion tips, which are always welcome, this is a new line in Entsophyian prejudices (Entsophisms, I call them). Hereabouts it’s too cool for bare feet and too warm for shoes, so please lay off us sandals-with-socks people. I have enough hang-ups with not being a card-carrying Jaynesian Bayesian.

      • In retaliation, you could always write a paper on how Kuhn’s responsible for people over the age of 22 using the word “redonkulous”. ;-)

        • Nick: If society can forgive murderers after they’ve paid their debt to society, then there’s at least a 50/50 chance they’ll forgive sandals-with-socks.

          Corey: you know how to spell “redonkulous”?

        • Also Nick, whoever created the Universe clearly intended for men to wear taylored suits (except when they’re blowing stuff up – oorah Marines!) and for women to wear little black dressed (except when they’re blowing stuff up – oorah lady Marines!). Since I had nothing to do with Creation, you can’t blame that on me.

  2. Entsophy: Sure saying the sign “empty gas cans” caused the factory to burn down would be silly, but arguing that sign’s tendency to suggest less dangerous when in fact empty gas cans are far more volatile than full ones – is sensible.

    By just looking at the other semioticians he is quoting, I just recognise Peirce whom the author does know something about.

    Andrew might like this quote “Peirce, who well realized that novel scientific hypotheses can hardly be assigned probabilities at all” re: not being interested in posterior probabilities of hypotheses.

    I need to read something from semiotics this summer sometime anyways.

  3. It’s important to separate clearly between fraud and every-day normal sucky science. Not only because there’s a clear difference regarding the individual, but also because of the expected impact. How many people actually fake stuff? Probably not many. We will catch some, won’t catch some, in the end, they’re adding a bit of noise. How many people in the social sciences and elsewhere use methods that aren’t quite fraud, but are, I think the standard term is “questionable”? Some days it feels as if the right answer is damn near everyone. That’s a LOT of added noise. How many of them feel guilty about it? Some; not all; possibly only a minority. Because we think what we’re doing is fine. So we’re calculating our p values based on a sample of 21 and happily report our post-hoc stories whenever we massage something until it manages to limbo towards publications by passing the .05 bar, and feel mighty proud of ourselves because we applied Bonferroni and tested for normality.
    And just being angry at the actual fraudsters might give everyone who’s doing questionable, but non-criminal stuff a good conscience, because after all, our data wasn’t simply made up – just a bit dressed up.

  4. I got baited into this, so let me just qualify the following obnoxiousness by saying 1) I have learned a lot from Kuhn and Quine; 2) I know much much less about Philosophy of Science than Dr. Pietarinen; and 3) I hope he will forgive me for trashing the rhetorical quality of his argument on the internet. But I can’t help myself, and I’m pretty sure its fair game to be a jerk about a paper someone puts on the internet..

    An annotated outline of the paper and argument, with scoring (subjective brownie points for thought(+)/nonsense(-) in parentheses).

    Thesis: Academic fraudster is a natural result of anti-realist philosophy of science and/or relativist epistemology.

    First Point: This guy named Stapel was a cheat and a liar and sucked (+1)
    Supporting points: Guy very good at “theory” (+/- 0 — just a set-up)
    History of trial (+/- 0 — just waiting space recapping in unnecessary detail)
    An incoherent argument (possibly due in part to a couple of typos/word omissions) on how this isn’t normal statistical practice (something about when you can/t exclude data) and ends with “What was this guy thinking about?” (-1 for time wasting, lack of coherence of paragraph 2 on page 3).

    Transition: That last question was a psychological one. But I don’t care about that. I will say he was criminally minded, and then consider only the philosophic grounding which may have either induced (hard to know why) or abetted (hard to know why) his criminality. (-1 for WTF transition – here I would write a snarky note to my undergrad saying “Why did you wasted 2+ pages on background only to make this lousy transition to what you are actually interested in).

    Second Point: Scientific fraud is a symptom of bad (anti-realist, relativist, simplistic) philosophy of science. (-1 for “wait really? because a guy you say is criminally-minded cheated on something?” +1 for use of the word “posse” in a pseudo-academic work).

    Supporting Points: Stapel admitted that our scientific claims are under-determined (+1 for “oh ish, that actually seems almost relevant).
    Kuhn (or stupid people reading Kuhn) thought science was basically a sociological construct where certain rules determined what was true (a Foucaultian Kuhn, but OK). Here’s why this thinking is wrong (-1 for I don’t need a critique of a theory of Kuhn here, I need to see the theory itself and how it would lead to justification of fraud).
    Now we spend a couple pages on Kuhn’s thinking, but mostly on how he was either mis-interpreted (but without giving any real substance on a better interpretation), and being as dismissive as possible of his contributions by attributing the important stuff to someone else ( -2 for “I’m so bored right now who cares tie this into the thesis please”)
    “Another idea in which social factors have been proposed to enter the formation of scientific
    beliefs is the underdetermination thesis.” (-3: -1 for undergrad quality transition; -2 for “but you never tied the last 3 pages to the thesis in any meaningful way.
    Now a page and a half on under-determination, that doesn’t really explain the philosophical arguments, just states the author’s position with regards to them. (-2 for seriously get to the point please).

    Now there are 3 stars on the page, which I believe means a section break, but don’t show up anywhere else in the paper. Maybe this just means “I’ve given a bunch of background of a) a fraudulent jerk and b) two things that piss me off in the philosophy of science. Now I will tie these things together” (+1 for OMG maybe the point is coming soon!; -2 for I know you’re just getting my hopes up for no reason)

    Third Point: “Needless to say, teaching philosophy of science is vitally important to achieve the goals of
    general science education” (-5 WTF WTF WTF?)

    Supporting point(s): OK. I think this is the entire “substance” of his argument:

    “But it is neither the resentment nor envy of science that keeps an honest inquirer busy at work. It is the potentiality of hypotheses turning out false, however inconvenient or painful that may be. Losing sight of the truth guiding the inquiry may deliver wrong causes unworthy of fighting for. Though truth in the limit is something never actually reached, its dismissal is unsafe.”

    Interpretation (generous): A good scientist wants to push his theory hard, even if that means s/he was wrong. But bad scientists just want to be right. Though we may never get to truth, we have to strive towards it. ???? . Kuhnian/Quinian reasoning leads people (somehow) to be bad scientists, because “truth isn’t real!”)

    The ???? part seems to relate the “scientist who just wants to be right” with some anti-realist philosophy. He seems to assume something like “if you don’t believe in Truth, you will become an academic fraud”. And now I’m kinda seeing the point. The point is – If you don’t FEAR G-D you will probably go out and murder your neighbor, because you are an atheist, so why not do it? (-7 for “because you can play by the rules even if you think its just a game people invented.”).

    Fourth Point (is this a point at all?): Digression about statistical findings: “Now in applied statistics, it is not too difficult to find dependencies from almost any dataset (+1). However, only very few of these dependencies are true (-1, huh? what kind of “true”?). By the very nature of the way the world is, most of them will be false (-3 for huh? Do you mean a causal effect, or a correlation, or what? Because I think a lot of correlations in the world a “real”, just not very informative).” (-8 more for making a point about 70 times less well than Andrew Gelman would make it and without any statistical/analytic/conceptual rigor).

    Supporting point: This kind of publishing of correlations and nonsense happens because we forgot Popper and testing theories and whatnot (-1 for no nuance/detail where you want it). And because of incentives (-1 for banal). (-5 more for “shouldn’t you relate the incentives for publishing crap with an anti-realist philosophy and show how these things together give rise to the academic fraudster?”)

    Conclusion: “In the light of the scams of the magnitude of the Stapel Case, I here raise the issue of whether the relativist ideas – which originally might have looked as much more innocent playgrounds for those who did not suspect the worst – have now shown their grotesque sides in the proliferation of pseudo-inquiry.”
    (+5 for honesty: “I hear raise the issue” is exactly what he does. Then he talks around it, makes some totally irrelevant claims, discusses minutia we don’t care about regarding philosophical battles without relating them to the thesis at hand, throws in some stats jargon, and re-asserts his thesis).
    (-100 for “Where the hell was the argument here about how anti-realist/relativist positions lead to fraud”?)

    • “By the very nature of the way the world is, most of them will be false (-3 for huh? Do you mean a causal effect, or a correlation, or what? Because I think a lot of correlations in the world a “real”, just not very informative).”

      This comes from the overwhelming desire most people have to interpret all data as coming from a population or urn whose properties are described by a frequency distribution. This is fine if there really is a population or urn that the data was drawn from, but most of the time it’s a complete fantasy not even approximately related to anything in the real world.

      The bizarre result is that instead of thinking data=real and population=fantasy they start to really believe the data is some kind of amorphous phantom, while the made up “population” is a hard fact.

      It is so much more down to earth to say the correlations in the data were real (obviously) and to simply ask what, if anything, they imply about other facts/data we might care about.

  5. jrc:

    As you put it “we don’t care about regarding philosophical battles”

    This is why I will read it but not until I am bored and need something to read ;-)

    The author’s intended audience likely only cares about that!

    Can _we_ get something out of reading it?

    As the author only seemed to reference Peirce as a semiotician I can guess at the underlying argument.

    Peirce put it one a scientific community is based on faith, hope and charity.

    Faith that scientific method will get you less wrong.

    Hope that the community will continue to exist.

    Charity in that you forgo individual advantage for the sake of the community (i.e. don’t do those small studies using mech. Turk. and highly promote those with p_value < .05 or as Don Rubin pointed out at JSM2013 casually change an RCT interim analysis plan to take advantage of the huge fuss raised by poorly done observational studies that _identified_ a risk with the treatment)

    Some nominalist and relative philosophies consider individuals and their interests as paramount so won't accept forgoing personal advantage (by doing things like checking their prior).

  6. There is very little engagement with Kuhn’s thinking in this paper. I would be very interested to hear Pietarinen explain how I can hold a piece of undescribed reality in one hand and a description in the other and know whether or not they are in agreement. We are just supposed to be scared into thinking that if we don’t keep chanting the vacuous slogan “truth is correspondence with reality!” we ought to expect people to just make up their data.

  7. “So what on earth was this guy thinking about? I am not very interested in this psychological question.”

    I think that is unfortunate. Stapel strikes me as a pretty classic conman. Psychology seems more relevant than philosophy here.

    You can see the impact of philosophies and politics on the numerous campaigns to shut down politically incorrect researchers, or in the overemphasis and lack of skepticism given to certain politically correct findings (e.g., stereotype threat). But Stapel wasn’t putting his thumb on the scale, he was just making up all his data. He’s more like Bernie Madoff or Stephen Glass.

  8. Knocking this paper sounds too much like fun. @jrc’s deconstruction is a classic of its kind; it’s a pity that conventions and attitudes probably make it unpublishable, but it’s unlikely even to be matched.

    So let’s try a positive. Fraud is clearly a big issue and it is an issue for philosophy of science, approached any way one wants, of whether and how it fits in to philosophy of science. Here we have one attempt to think it through, unconvincing though it seems.

    Unfortunately, the “whether” question can be answered No — psychology, sociology, criminology even seem more apposite in casting real light on fraud, as others have rightly remarked — but there is not much of a paper in that for a philosopher of science.

    I feel sorry for philosophy of science. Back in the day — 1960s, 1970s, … — it seemed quite exciting and important even to fairly ordinary scientists and social scientists with a modicum of curiosity about foundational or fundamental questions. They (we!) could feel with very modest effort that they grasped the gist of what Popper, Kuhn and lesser characters were saying and that there were key debates that could be gone through with respect to your own corner of science or social science.

    No doubt this irritated the heck out of most of the professionals, who from their standpoint could see mostly oversimplifications of the simplified end of their field. The awkward tensions between history of science and philosophy of science were also often overlooked by amateurs outside (history and) philosophy of science.

    But the field seems to have spent most of the last few decades saying “It’s more complicated than that”, which is all too true but much more tedious. What’s the exciting stuff that we should all be reading now? Philosophy of science seems to have passed its peak of external popularity; whether fairly or not some others here are in a better position to judge.

    • Nick:

      I gave a talk at Oxford Stats Dept around ten years ago discussing this challenge of realising every statistician needs some sense of philosophy of science (even if just to avoid making a fool of themselves by suggesting statistics provides a grammar of science – e.g. Pearson)

      Probably quoted Dewey about bringing the best of philosophy to bear on [statistical] everyday problems rather than turning [statistical] everyday problems into problems of philosophy.

      But how does anyone get any good sense of philosophy of science without first drowning in the swampy cesspool of literature that one has to wander into.

      No answers yet.

      p.s. OK maybe more than a few one eyed kings whom offer to lead us through safely

      • O’Rourke,

        My impression from reading the papers and books of the great mathematical physicists is that they treated philosophies in a way reminiscent of the way Gelman/Jaynes treat statistical models: take then very seriously, see exactly what they imply, and then move on.

        Similarly, The greats seemed to have had the habit of taking a philosophical viewpoint very seriously and really using it to see where it lead, and then dropping it for another one. Consequently both people who believe “real scientists ignore philosophy” and people who believe “real scientists are heavily influence by philosophy” can find often evidence for their respect viewpoints in the works of the same scientists. Scientist who took their philosophy too seriously like Bohr for example, haven’t worked out that great. Those who make a pretense of rejection all philosophy usually never do anything so we don’t know their names today.

        The reduction of philosophy to a profession is joke however. They should immediately change philosophy department’s name to “The History of Philosophy” since that’s what they really are. There’s not a shred of evidence professionalizing philosophy was a good idea. It should just be an avocation open to anyone, including Historians of Philosophy if they like, but in which being a Historian provides no special boost over anyone else. Any serious philosopher should have a day job.

        • “Any serious philosopher should have a day job.”

          Does sound like a good idea, this being forced to regularly deal with the empirical world and run into _brute force reality_ that makes it very clear you are wrong again.

  9. Stapel cheated because it was easy to do and he could get away with it. He didn’t even do it in a very sophisticated way – for example, he recycled entire datasets (which he’d built by hand), cell-by-cell from one study to the next, rather than do something moderately sophisticated like setting up Excel to generate reasonably normally-distributed random numbers with appropriate means and SDs. This was one of the things that led to him being caught. (By the way, he did plenty of non-fraudulent studies too, and Pietarinen’s throwaway remark that Stapel probably faked his PhD seems unlikely; Stapel states that he handed his PhD back because he felt he hadn’t lived up to the standards expected of him, and he has no reason to lie about that.)

    In his book “Ontsporing” (which I hope will appear in English at some point), Stapel describes the fear he felt when discussing his fraudulent work in meetings with colleagues that someone would say “Diederik, for your data to be true requires the earth to be flat”. But nobody ever called him out on it. The psychology (ha!) of how that sort of thing happens would be a fascinating subject for study. But it’s basically old-fashioned greed and laziness, taking place in an environment that was not administratively/bureaucratically set up to catch it, not some kind of indication that the theoretical roof is falling in.

  10. I can’t speak for this author, but I’ve often had the thought that Stapel is the perfect exemplar of pure deconstructionism/radical social-constructivism. That is, the more extreme interpretations of some of Kuhn’s theses—ones Kuhn never quite denies (perhaps because his fame was built upon them). Stapel makes this fairly clear in his book and in speeches: it’s all a matter of selling what people want to believe. There’s more than a little influence: the very terminology is Kuhn’s, and the Netherlands has always gravitated toward the more extreme post-modern themes in philosophy.
    The following is from the NYT interview: http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all&_r=0
    “Several times in our conversation, Stapel alluded to having a fuzzy, postmodernist relationship with the truth, which he agreed served as a convenient fog for his wrongdoings. “It’s hard to know the truth,” he said. “When somebody says, ‘I love you,’ how do I know what it really means?” …“People have lost faith in the church, but they haven’t lost faith in science. My behavior shows that science is not holy.”
    What the public didn’t realize, he said, was that academic science, too, was becoming a business. …I am a salesman. I am on the road. People are on the road with their talk. With the same talk. It’s like a circus.”

    From my blog on Stapel:
    http://errorstatistics.com/2013/06/22/what-do-these-share-in-common-mms-limbo-stick-ovulation-dale-carnegie-sat-night-potpourri/

    http://errorstatistics.com/2013/07/03/philstatlaw-50-shades-of-gray-between-error-and-fraud/

    Separately, the Bayesian model in philosophy was/is regarded by many as a way to cope with the challenges Kuhn brought to logical empiricism. It was thought/hoped (by such hard-headed philosophers of science as Wesley
    Salmon) that one could incorporate Kuhnian factors into science while not robbing it entirely of having a logic based in part on empirical evidence. Kuhn, however, explicitly denied there was any argument for supposing a shared algorithm of the Bayesian or any other sort (in science). There was a big session at the Amer Philo Assoc where Kuhn met with Salmon and Hempel!! Well, this is too long already…

    • Mayo: Do you consider a discussion of this at top level (i.e., not just referencing in the comments section) in your blog? I think it would be interesting.

      General: I think that constructivism is mis-represented by sentences like “it’s all a matter of selling what people want to believe”. Stapel sees very clearly that his fabricated data deviate from “real observed data” and that “real observed data” could have been obtained by really carrying out the experiments. Therefore, Stapel is a fraudster even according to his own *personal* (constructed) reality, and constructivists and realists can be concerned about research ethics in pretty much the same way here.

    • Exactly. Plus it just shuts down debate since it will convince no one who isn’t already convinced and all anyone can say to counter the slander is “I don’t believe it”. So debate ends. Better to just attack Kuhn directly in a stand up fight.

  11. Let’s keep in mind that Stapel never argued based on some corrupt philosophy of science that the data don’t matter. He took for granted that they do as evidenced by the fact that he lied about his data. Dishonest scientists predate Kuhn, and dishonesty is not supported by Kuhnian or any other philosophy of science that I ever heard of.

    • Absolutely, and in that sense I disagree with the article that suggests that it’s all those pesky Kuhnians. In a perverted sense, however, the current seeking of splashy results and damn the torpedoes may be what happens when the goal of seeking “revolutionary” science meets scientific administration metrics….

      • Isn’t successfully faking radical paradigm-shattering science harder than faking normal science. All of the results that Stapel found are in the tradition of priming-based studies of psychology that, while super trendy and interesting, are clearly normal science. There are so many other similar results with such priming experiments; who will doubt your faked data.

        • Yes, it’s harder to fake truly revolutionary work.

          The reason why I called it “perverted Kuhnianism” in the sense that that it has morphed “revolutionary” into “counterintuitive” and “likely to make for good press.” These two are much more readily faked and easily quantified aspects of truly revolutionary work.

          The notion of “perverted” I took from Aristotle, so the example of a perverted democracy has the easily observed attributes of a democracy such as voting, a constitution, or offices like a president, prime minister, etc. But of course, if power isn’t actually exercised in a manner consistent with the way that the populace votes, e.g., if the equivalent of Augusto Pinochet always makes sure that things that threaten him don’t come onto the agenda. Other examples include perverting the voting process by making eligibility systematically exclude entire classes of voters (hi North Carolina). This sort of thing happens in science when the review process is distorted by one or more big names who prevent others from getting grants or publishing.

          Note: Aristotle referred to democracy as one of the perverted forms. I’m not quoting him directly, but the general idea. ;)

    • Well I think he did allude to a social-constructivism/post-modern philosophy. I’m guessing most people on this blog haven’t had a lot of exchanges with the “extremists” of which their are entire departments (e.g., in STS), and with which philosophers regularly deal. They will say, “of course we do not support dishonesty,” that would entail there being such a thing as honesty or truth beyond whatever you can negotiate in the relevant political-socio-psych-econ group you are in. That just IS what’s really authentic. To deny that that is self-deception—according to those who have latched onto this philosophy. I’m not exaggerating in the slightest.

      • Despite what I do for a living these days, working as a psychometrician in an educational psychology department, in my misspent youth I knew quite a number of post-modern toasties that were just as you describe. Usually they were more interested in Sticking It To The Man than in what I would think of as genuine inquiry. Unfortunately the intellectual habits have begun to inhabit the right these days, too. :/

        Learning a bit of Foucault is a good thing and more statisticians, psycho/bio/econometrics folks, and that general ilk would benefit from some exposure, as Ian Hacking would likely attest. It keeps you honest and helps realize that your point of view isn’t naively true or neatly separate from power arrangements. But beware staring too long into that particular abyss….

      • The concern that if we don’t endorse a belief in a mind-independent reality as a core scientific value then we should expect people to give up on other values such as honesty and the pursuit of making warranted assertions just sounds like straw man fear-mongering to me. I doubt that there are really such people, and if there were we wouldn’t have to take them seriously. If someone claimed to be a scientist who did not value honesty I wouldn’t believe her.

  12. I was not sympathetic to the author’s accusation before reading the paper, and I am not more sympathetic now that I have failed to agree with the first part and failed to understand what the point of the second part was.

Comments are closed.