Skip to content
Archive of posts filed under the Miscellaneous Science category.

Carol Nickerson investigates an unfounded claim of “17 replications”

Carol Nickerson sends along this report in which she carefully looks into the claim that the effect of power posing on feelings of power has replicated 17 times. Also relevant to the discussion is this post from a few months ago by Joe Simmons, Leif Nelson, and Uri Simonsohn. I am writing about this because […]

Tools for detecting junk science? Transparency is the key.

In an article to appear in the journal Child Development, “Distinguishing polemic from commentary in science,” physicist David Grimes and psychologist Dorothy Bishop write: Exposure to nonionizing radiation used in wireless communication remains a contentious topic in the public mind—while the overwhelming scientific evidence to date suggests that microwave and radio frequencies used in modern […]

The all-important distinction between truth and evidence

Yesterday we discussed a sad but all-too-familiar story of a little research project that got published and hyped beyond recognition. The published paper was called, “The more you play, the more aggressive you become: A long-term experimental study of cumulative violent video game effects on hostile expectations and aggressive behavior,” but actually that title was […]

More bad news in the scientific literature: A 3-day study is called “long term,” and nobody even seems to notice the problem. Whassup with that??

Someone pointed me to this article, “The more you play, the more aggressive you become: A long-term experimental study of cumulative violent video game effects on hostile expectations and aggressive behavior,” by Youssef Hasan, Laurent Bègue, Michael Scharkow, and Brad Bushman. My correspondent was suspicious of the error bars in Figure 1. I actually think […]

Yet another IRB horror story

The IRB (institutional review board) is this weird bureaucracy, often staffed by helpful and well-meaning people but generally out of control, as it operates on an if-it’s-not-allowed-it’s-forbidden principle. As an example, Jonathan Falk points us to this Kafkaesque story from Scott Alexander, which ends up like this: Faced with submitting twenty-seven new pieces of paperwork […]

“It’s not just that the emperor has no clothes, it’s more like the emperor has been standing in the public square for fifteen years screaming, I’m naked! I’m naked! Look at me! And the scientific establishment is like, Wow, what a beautiful outfit.”

Somebody pointed Nick Brown to another paper by notorious eating behavior researcher Brian Wansink. Here’s Brown: I have that one in my collection of PDFs. I see I downloaded it on January 7, 2017, which was 3 days before our preprint went live. Probably I skimmed it and didn’t pay much further attention. I don’t […]

Gaydar and the fallacy of objective measurement

Greggor Mattson, Dan Simpson, and I wrote this paper, which begins: Recent media coverage of studies about “gaydar,” the supposed ability to detect another’s sexual orientation through visual cues, reveal problems in which the ideals of scientific precision strip the context from intrinsically social phenomena. This fallacy of objective measurement, as we term it, leads […]

Reasons for an optimistic take on science: there are not “growing problems with research and publication practices.” Rather, there have been, and continue to be, huge problems with research and publication practices, but we’ve made progress in recognizing these problems.

Javier Benitez points us to an article by Daniele Fanelli, “Is science really facing a reproducibility crisis, and do we need it to?”, published in the Proceedings of the National Academy of Sciences, which begins: Efforts to improve the reproducibility and integrity of science are typically justified by a narrative of crisis, according to which […]

A more formal take on the multiverse

You’ve heard of multiverse analysis, which is an attempt to map out the garden of forking paths. Others are interested in this topic too. Carol Nickerson pointed me to this paper by Jan Wacker with a more formal version of the multiverse idea.

Murray Davis on learning from stories

Jay Livingston writes: Your recent post and the linked article on storytelling reminded me of Murray Davis’s article on theory, which has some of the same themes. I haven’t reread it in a long time, so my memory of the details is hazy. Here are the first two paragraphs, which might give you an idea […]

Information flows both ways (Martian conspiracy theory edition)

A topic that arises from time to time in Bayesian statistics is the desire of analysts to propagate information in one direction, with no backwash, as it were. But the logic of Bayesian inference doesn’t work that way. If A and B are two uncertain statements, and A tells you something about B, then learning […]

“Like a harbor clotted with sunken vessels”

After writing this post on an error in one of my published papers, I got to thinking about the general problem of mistakes in the scientific literature. Retraction is not a serious solution to the problem. And there are lots of people out there who simply refuse to admit, let alone correct, their published errors: […]

Audition (Fools Who Explore)

This story inspired me to very slightly alter the Hurwitz, Pasek, and Paul classic: My aunt used to work in a lab I remember, she used to come home and tell us these stories about being abroad And I remember she told us that she jumped into the data once, barefoot She frowned Leapt, without […]

“Have Smartphones Destroyed a Generation?” and “The Narcissism Epidemic”: How can we think about the evidence?

Jay Livingston points to this hypey article, “Have Smartphones Destroyed a Generation?”, by Jean Twenge, who writes: I’ve been researching generational differences for 25 years . . . Typically, the characteristics that come to define a generation appear gradually, and along a continuum. . . . [But] Around 2012, I noticed abrupt shifts in teen […]

Low power and the replication crisis: What have we learned since 2004 (or 1984, or 1964)?

I happened to run across this article from 2004, “The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies,” by Scott Maxwell and published in the journal Psychological Methods. In this article, Maxwell covers a lot of the material later discussed in the paper Power Failure by Button et al. (2013), and the […]

Testing Seth Roberts’ appetite theory

Jonathan Tupper writes: My organization is running a group test of Seth Roberts’ old theory about appetite. We are running something like a “web trial” as discussed in your Chance article with Seth. And in fact our design was very inspired by your conversation… For one, we are using a control group which takes light […]

I’m skeptical of the claims made in this paper

Two different people pointed me to a recent research article, suggesting that the claims therein were implausible and the result of some combination of forking paths and spurious correlations—that is, there was doubt that the results would show up in a preregistered replication, and that, if they did show up, that they would mean what […]

“No System is Perfect: Understanding How Registration-Based Editorial Processes Affect Reproducibility and Investment in Research Quality”

Robert Bloomfield, Kristina Rennekamp, Blake Steenhoven sent along this paper that compares “a registration-based Editorial Process (REP). Authors submitted proposals to gather and analyze data; successful proposals were guaranteed publication as long as the authors lived up to their commitments, regardless of whether results supported their predictions” to “the Traditional Editorial Process (TEP).” Here’s what […]

Snappy Titles: Deterministic claims increase the probability of getting a paper published in a psychology journal

A junior psychology researcher who would like to remain anonymous writes: I wanted to pass along something I found to be of interest today as a proponent of pre-registration. Here is a recent article from Social Psychological and Personality Science. I was interested by the pre-registered study. Here is the pre-registration for Study 1. The […]

Geoff Norman: Is science a special kind of storytelling?

Javier Benítez points to this article by epidemiologist Geoff Norman, who writes: The nature of science was summarized beautifully by a Stanford professor of science education, Mary Budd Rowe, who said that: Science is a special kind of story-telling with no right or wrong answers. Just better and better stories. Benítez writes that he doesn’t […]