Skip to content
Archive of entries posted by

Authority figures in psychology spread more happy talk, still don’t get the point that much of the published, celebrated, and publicized work in their field is no good

Susan Fiske, Daniel Schacter, and Shelley Taylor write (link from Retraction Watch): Psychology is not in crisis, contrary to popular rumor. Every few decades, critics declare a crisis, point out problems, and sometimes motivate solutions. When we were graduate students, psychology was in “crisis,” raising concerns about whether it was scientific enough. Issues of measurement […]

I was gonna write a post entitled, “Unlocking past collaboration: student use affects mood and happiness,” but it didn’t seem worth the bother

Ivan Oransky points us to this hilarious story of a retracted paper in Psychological Science. The hilarious part is not the article itself (a dry-as-dust collection of small-N experiments with open-ended data-exclusion and data-analysis rules, accompanied by the usual scattering of statistically significant p-values in the garden) or even the reason for the retraction:

You can fit hidden Markov models in Stan (and thus, also in Stata! and Python! and R! and Julia! and Matlab!)

You can fit finite mixture models in Stan; see section 12 of the Stan manual. You can fit change point models in Stan; see section 14.2 of the Stan manual. You can fit mark-recapture models in Stan; see section 14.2 of the Stan manual. You can fit hidden Markov models in Stan; see section 9.6 […]

Theoretical statistics is the theory of applied statistics: how to think about what we do (My talk at the University of Michigan this Friday 3pm)

Theoretical statistics is the theory of applied statistics: how to think about what we do Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University Working scientists and engineers commonly feel that philosophy is a waste of time. But theoretical and philosophical principles can guide practice, so it makes sense for us to […]

The Mannequin

Jonathan Falk points to this article, “Examining the impact of grape consumption on brain metabolism and cognitive function in patients with mild decline in cognition: A double-blinded placebo controlled pilot study,” and writes: Drink up! N=10, no effect on thing you’re aiming at, p value result on a few brain measurements (out of?), eminently pr-able […]

The “What does not kill my statistical significance makes it stronger” fallacy

As anyone who’s designed a study and gathered data can tell you, getting statistical significance is difficult. Lots of our best ideas don’t pan out, and even if a hypothesis seems to be supported by the data, the magic “p less than .05” can be elusive. And we also know that noisy data and small […]

Long Shot

Frank Harrell doesn’t like p-values: In my [Frank’s] opinion, null hypothesis testing and p-values have done significant harm to science. The purpose of this note is to catalog the many problems caused by p-values. As readers post new problems in their comments, more will be incorporated into the list, so this is a work in […]

Research connects overpublication during national sporting events to science-journalism problems

Ivan Oransky pointed me to a delightful science-based press release, “One’s ability to make money develops before birth”: Researchers from the Higher School of Economics have shown how the level of perinatal testosterone, the sex hormone, impacts a person’s earnings in life. Prior research confirms that many skills and successes are linked to the widely […]

Death of the Party

Under the subject line, “Example of a classy response to someone pointing out an error,” Charles Jack​son writes: In their recent book, Mazur and Stein describe the discovery of an error that one of them had made in a recent paper writing: “Happily, Bartosz Naskreki spotted this error . . .” See below for full […]

Pizzagate, or the curious incident of the researcher in response to people pointing out 150 errors in four of his papers

There are a bunch of things about this story that just don’t make a lot of sense to me. For those who haven’t been following the blog recently, here’s the quick backstory: Brian Wansink is a Cornell University business school professor and self-described “world-renowned eating behavior expert for over 25 years.” It’s come out that […]

I’m thinking of using these at the titles for my next 97 blog posts

Where do you think these actually came from? (No googling—that would be cheating.) P.S. Anyone who wants to know the answer can google it. But there were some great guesses in the comments. My favorite, from Frank: I’ve got to go with “before the colon” in questionable social science papers, e.g: “Don’t make me laugh: […]

When do protests affect policy?

Gur Huberman writes that he’s been wondering for many years about this question: One function of protests is to vent out the protesters’ emotions. When do protests affect policy? In dictatorships there are clear examples of protests affecting reality, e.g., in Eastern Europe in 1989. It’s harder to find such clear examples in democracies. And […]

Criticism of bad research: More harm than good?

We’ve had some recent posts (here and here) about the research of Brian Wansink, a Cornell University business professor who’s found fame and fortune from doing empirical research on eating behaviors. It’s come out that four of his recent papers—all of them derived from a single experiment which Wansink himself described as a “failed study […]

No guru, no method, no teacher, Just you and I and nature . . . in the garden. Of forking paths.

Here’s a quote: Instead of focusing on theory, the focus is on asking and answering practical research questions. It sounds eminently reasonable, yet in context I think it’s completely wrong. I will explain. But first some background. Junk science and statistics They say that hard cases make bad law. But bad research can make good […]

Washington Post falls for that horrible air-rage study, and what gets me really angry about this

Someone just pointed me to this news article entitled, “Air rage incidents are on the rise. First-class sections aren’t helping,” which falls hook, line, and sinker for a notorious discredited study that appeared in PPNAS last year. I can hardly blame the Washington Post reporter for getting this one wrong, given that NPR swallowed the […]

Age period cohort brouhaha

Hi everybody! In August, I announced a break from blogging. And this is my first new post since then. (not counting various interpolated topical items on polling, elections, laughable surveys comparing North Carolina to North Korea, junk science on pizza prices, etc) I’m still trying to figure out how to do this; I have a […]

How to attack human rights and the U.S. economy at the same time

I received this email from a postdoc in a technical field: As you might have heard, Trump signed an executive order today issuing a 30-day total suspension of visas and other immigration benefits for the citizens of Iran and six other countries. For my wife and me, this means that our visas are suspended; we […]

Absence of evidence is evidence of alcohol?

Arho Toikka writes: I ran across what I feel is a pretty peculiar use of statistical significance and p-values, and thought I’d send you a message and see if you find it interesting too or if I’m just confused about something: I read a news story about a study that showed that previous studies on […]

“Statistical heartburn: An attempt to digest four pizza publications from the Cornell Food and Brand Lab”

Tim van der Zee​, Jordan Anaya​, and Nicholas Brown posted this very detailed criticism of four papers published by food researcher and business school professor Brian Wansink. The papers are all in obscure journals and became notorious only after Wansink blogged about them in the context of some advice he was giving to graduate students. […]

Let’s try to understand our own contradictions (what P. J. O’Rourke gets but Michael Lind doesn’t)

One way to understand the limitations of our own political attitudes is to recognize that other people think differently. Not just oppositely, but differently. To put it algebraically, suppose you believe A, B, C, D, E. OK, you won’t be surprised to hear that some people believe not-A, not-B, not-C, not-D, not-E. These are the […]