Skip to content
 

It’s all about Hurricane Andrew: Do patterns in post-disaster donations demonstrate egotism?

Jim Windle points to this post discussing a paper by Jesse Chandler, Tiffany M. Griffin, and Nicholas Sorensen, “In the ‘I’ of the Storm: Shared Initials Increase Disaster Donations.”

I took a quick look and didn’t notice anything clearly wrong with the paper, but there did seem to be some opportunities for forking paths, in that the paper seemed to be analyzing only a small selection of relevant data on the question they were asking.

I wrote that I’m open to the possibility that this is real, also open to the possibility that it’s not.

Windle replied:

That was my take as well. Human psychology is certainly strange enough that its possible, but human psychology is strange enough to allow seeing effects where there are none.

Well put.

The person I’d really want to ask about this one is Uri Simonsohn. He’s the one who wrote that paper several years ago carefully shooting down every claim from the dentists-named-Dennis article.

15 Comments

  1. Paul Alper says:

    Andrew wrote:

    “The person I’d really want to ask about this one is Uri Simonsohn. He’s the one who wrote that paper several years ago
    carefully shooting down every claim from the dentists-named-Dennis article.”

    However, Simonsohn’s frequent coauthors are Leif Nelson and Joseph Simmons. Those two wrote in 2007, “Moniker Maladies: When
    Names Sabotage Success.” In this paper Nelson and Simmons assert “there is a name-letter effect which may negatively affect
    the individual’s success in life.”

    The first claim in that paper is that baseball players named Kenny, Karl or Kasper are more likely to strike out than people
    who don’t have an initial K in their name because “K” is the symbol used to record strike outs. Nelson and Simmons looked at
    6397 Major League Baseball players and found that “batters whose names began with K struck out at a higher rate (in 18.8% of
    their plate appearances) than the remaining batters (17.2%).” This assertion turns out to be not only unbelievable but false
    empirically. Using the same database, a blogger at http://sabermetricresearch.blogspot.com/ found otherwise; in particular,
    for 1960s to 2000s: Ks 14.5%, non-Ks 14.2%. This blogger concludes with, “So the big question remains: why did the authors
    get such a high strikeout rate difference?” The answer is that Nelson and Simmons weighted each player equally when in fact,
    there is a vast discrepancy in plate appearances.

    Nelson and Simmons second claim–there are three more–is “As predicted, students whose names begin with a C or D earned
    lower GPAs [grade point averages] than students whose names begin with A or B, F(4, 14348) = 4.55” yielding a p-value of
    “.001.”

    With that in mind, here is the last sentence of the paper’s abstract:”These findings provide striking evidence that
    unconsciously desiring negative name-resembling performance outcomes can insidiously undermine the more conscious pursuit of
    positive outcomes.”

    • psyoskeptic says:

      “However, Simonsohn’s frequent coauthors are…”

      While the article is related to this topic, this looks like an ad hominem attack on Simonsohn through his co-authors using a paper from before he had anything published on research methods. Maybe it was before they had even met.

      Some people learn and change. Simonsohn’s own pre False Psychology publications and post are very different. Most of the pre ones look very much like something he’d probably condemn.

      • Joe Simmons says:

        Leif and I have publicly admitted that our Moniker Maladies paper is almost certainly a false-positive result (http://datacolada.org/53/). Indeed that paper was published before we realized the implications of p-hacking. We learned and we changed.

        • Paul Alper says:

          Joe Simmons wrote:

          “Indeed that paper was published before we realized the implications of p-hacking. We learned and we changed.”

          Simmons in this paper (http://datacolada.org/53/) states

          “For example, we set out to show that (not: test whether) people with an A or B initial get better grades than people with a C or D initial. After many attempts (we ran many analyses and we ran many studies), we found enough “evidence” for this hypothesis, and we published the findings in Psychological Science. At the time, we believed the findings and this felt like a success. Now we both recognize it as a failure.”

          But Why in the world would anyone believe this worthy of investigation in the first place? To answer this question, consider what http://skepdic.com/toothfairyscience.html says about Tooth Fairy science:
          ———
          “Tooth Fairy science” is an expression coined by Harriet Hall, M.D., (aka the SkepDoc) to refer to doing research on a phenomenon before establishing that the phenomenon exists. Tooth Fairy science is part of a larger domain that might be called Fairy Tale science: research that aims to confirm a farfetched story believed by millions of scientifically innocent minds. Fairy Tale science uses research data to explain things that haven’t been proven to have actually happened. Fairy Tale scientists mistakenly think that if they have collected data that is consistent with their hypothesis, then they have collected data that confirms their hypothesis. Tooth Fairy science seeks explanations for things before establishing that those things actually exist. For example:

          You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists.
          ———

          • Andrew says:

            Paul:

            The short answer to your question is that people thought that their hypotheses had confirmation. In a typical psychology paper, there might be 5 or 10 separate studies, each independently assessing a certain hypothesis. Performing a series of such studies gave a sense of confirmation. In retrospect, this sense was mistaken, but it took awhile for people to realize this.

            • Paul Alper says:

              Andrew said: “In retrospect, this sense was mistaken, but it took awhile for people to realize this.”

              Already in 2001, Leibovici wrote the famous “Effects of remote, retroactive intercessory prayer on outcomes in patients with bloodstream infection” and is must reading because

              “The purpose of the article was to ask the following question: would you believe in a study that looks methodologically correct but tests something that is completely out of people’s frame (or model) of the physical world.”

              Returning to baseball strikeouts and the letter “K”, why not the forking path of “W” because of Whiffing, leading to William, Willie and even Bill, or the F-word, Fanning, as in Frank, Frankie, Freddie, and even Phil. Data dredging can run deep. And while we are near the bottom, my last name means vain and lazy in the Basque language. Fortunately, I learned this late in life from my father-in-law so it had no effect on my childhood.

              • > Fortunately, I learned this late in life…so it had no effect on my childhood.

                except perhaps a retroactive intercessory effect which we should clearly test using a randomized controlled trial ;-)

          • Ethan Bolker says:

            But the tooth fairy does exist as a phenomenon (if not as an agent) and may indeed pay off more for first or last teeth. It’s probably not worth finding out, but it might be find-outable (though data would be very noisy). Then you’d have learned something.

          • psyoskeptic says:

            So, it’s clear that this is purely an ad hominem argument about association with people that don’t exist (the younger versions of these researchers).

            I just want to note that it’s not worth anyone’s time or attention and case they feel a need to be defensive at all.

  2. Elin says:

    That tooth fairy discussion is really stupid … basically the complaint from what I can tell is failure to put the first use of the term “tooth fairy” in quotation marks and define it as parents putting money under a pillow? Because while not my cup of tea it is perfectly legitimate to study lots of things about that cultural practice. It’s really that so many of these folks are terrified by the idea of society and that social behaviors follow patterns. The whole question of how much money to leave is very interesting and social, since the price has gone up over the years. It would indeed be interesting to know (for example) whether immigrant families adopt this tradition and how they learn about it (and whether they adapt it in some way to their home country culture). But oh no, the social is not “real” and has no “real” consequences even though all those kids are getting money and all those parents are figuring out how to be competitive in the tooth pricing market so their children are not shamed by their peers.

    • Andrew says:

      Elin:

      Just to clarify: I never used the term “tooth fairy” in this discussion, nor have I ever said the social is not “real.”

    • Anoneuoid says:

      I think the point is that they are measuring something other than what is assumed.

      My favorite example is a chemotherapy that reduces tumor growth via inducing nausea and thus caloric restriction, rather than whatever supposed mechanism. The drug really could “work”, but if you understood what was going on you would never inject poison into people just to get them eating less.

      • Carlos Ungil says:

        Your chemotherapy example is a rhetorical device or is it real?

      • Right, the point is that if you think the tooth fairy really exists, and you examine it scientifically, you will attribute all sorts of facts to an actual winged fairy. This has direct analogs to other areas of research where people claim “elderly words trigger slow walking” or whatever but there’s an assumed mechanism and so all the observed stuff is attributed to that mechanism without ever examining alternative mechanisms. In the tooth fairy example the point is if you never figure out that it’s the parents, you’ll never figure out even what you’re studying, and all the questions about things like whether immigrant parents adopt the practice will get confused with whether winged fairies discriminate against immigrant children or whatever the equivalent mis-attribution would be.

        The tooth fairy example is not stupid IMHO you simply moved past the important point because you know what the real mechanism is.

Leave a Reply