Skip to content
Archive of posts filed under the Miscellaneous Science category.

To understand the replication crisis, imagine a world in which everything was published.

John Snow points me to this post by psychology researcher Lisa Feldman Barrett who reacted to the recent news on the non-replication of many psychology studies with a contrarian, upbeat take, entitled “Psychology Is Not in Crisis.” Here’s Barrett: An initiative called the Reproducibility Project at the University of Virginia recently reran 100 psychology experiments […]

Uri Simonsohn warns us not to be falsely reassured

I agree with Uri Simonsohn that you don’t learn much by looking at the distribution of all the p-values that have appeared in some literature. Uri explains: Most p-values reported in most papers are irrelevant for the strategic behavior of interest. Covariates, manipulation checks, main effects in studies testing interactions, etc. Including them we underestimate […]

New paper on psychology replication

The Open Science Collaboration, a team led by psychology researcher Brian Nosek, organized the replication of 100 published psychology experiments. They report: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the […]

“Women Respond to Nobel Laureate’s ‘Trouble With Girls'”

Someone pointed me to this amusing/horrifying story of a clueless oldster. Some people are horrified by what the old guy said, other people are horrified by how he was treated. He was clueless in his views about women in science, or he was cluelessly naive about gotcha journalism. I haven’t been following the details and […]

“We can keep debating this after 11 years, but I’m sure we all have much more pressing things to do (grants? papers? family time? attacking 11-year-old papers by former classmates? guitar practice?)”

Someone pointed me to this discussion by Lior Pachter of a controversial claim in biology. The statistics The statistical content has to do with a biology paper by M. Kellis, B. W. Birren, and E.S. Lander from 2004 that contains the following passage: Strikingly, 95% of cases of accelerated evolution involve only one member of […]

Richard Feynman and the tyranny of measurement

I followed a link at Steve Hsu’s blog and came to this discussion of Feyman’s cognitive style. Hsu writes that “it was often easier for [Feynman] to invent his own solution than to read through someone else’s lengthy paper” and he follows up with a story in which “Feynman did not understand the conventional formulation […]

Psych dept: “We are especially interested in candidates whose research program contributes to the development of new quantitative methods”

This is cool. The #1 psychology department in the world is looking for a quantitative researcher: The Department of Psychology at the University of Michigan, Ann Arbor, invites applications for a tenure-track faculty position. The expected start date is September 1, 2016. The primary criterion for appointment is excellence in research and teaching. We are […]

“Physical Models of Living Systems”

Phil Nelson writes: I’d like to alert you that my new textbook, “Physical Models of Living Systems,” has just been published. Among other things, this book is my attempt to bring Bayesian inference to undergraduates in any science or engineering major, and the course I teach from it has been enthusiastically received. The book is […]

Economists betting on replication

Mark Patterson writes: A bunch of folks are collaborating on a project to replicate 18 experimental studies published in prominent Econ journals (mostly American Economic Review, a few Quarterly Journal of Economics). This is already pretty exciting, but the really cool bit is they’re opening a market (with real money) to predict which studies will […]

Humility needed in decision-making

Brian MacGillivray and Nick Pidgeon write: Daniel Gilbert maintains that people generally make bad decisions on risk issues, and suggests that communication strategies and education programmes would help (Nature 474, 275–277; 2011). This version of the deficit model pervades policy-making and branches of the social sciences. In this model, conflicts between expert and public perceptions […]

Sam Smith sings like a dream but he’s as clueless as Nicholas Wade when it comes to genetics

Psychologists speak of “folk psychology” or “folk physics” as the intuitive notions we have about the world, which typically describe some aspects of reality but ultimately are gross oversimplifications. I encountered a good example of “folk genetics” the other day after following the clickbait link to “22 Things We Learned Hanging Out With Sam Smith”: […]

Born-open data

Jeff Rouder writes: Although many researchers agree that scientific data should be open to scrutiny to ferret out poor analyses and outright fraud, most raw data sets are not available on demand. There are many reasons researchers do not open their data, and one is technical. It is often time consuming to prepare and archive […]

Cross-validation != magic

In a post entitled “A subtle way to over-fit,” John Cook writes: If you train a model on a set of data, it should fit that data well. The hope, however, is that it will fit a new set of data well. So in machine learning and statistics, people split their data into two parts. […]

My final post on this Tony Blair thing

Gur Huberman writes on the recent fraud in experiments in polisci: This comment is a reaction to the little of the discussion which I [Gur] followed, mostly in the NYTimes. What I didn’t see anybody say is that the system actually worked. First, there’s a peer-reviewed report in Science. Then other people deem the results […]

The greatest impediment to research progress is not impediments to research progress, it is scientists reading about impediments to research progress

My short answer is that I think twitter is destructive of clear communication. Now I’ll give the question, and I’ll give my long answer. Here’s the question provided by a reader: Just wondering what you thought of Brian Nosek’s recent comment on twitter, “The biggest impediment to research progress is not fraud, it is all […]

John Bohannon’s chocolate-and-weight-loss hoax study actually understates the problems with standard p-value scientific practice

Several people pointed me to this awesome story by John Bohannon: “Slim by Chocolate!” the headlines blared. A team of German researchers had found that people on a low-carb diet lost weight 10 percent faster if they ate a chocolate bar every day. It made the front page of Bild, Europe’s largest daily newspaper, just […]

Creativity is the ability to see relationships where none exist

Brent Goldfarb and Andrew King, in a paper to appear in the journal Strategic Management, write: In a recent issue of this journal, Bettis (2012) reports a conversation with a graduate student who forthrightly announced that he had been trained by faculty to “search for asterisks”. The student explained that he sifted through large databases […]

Can talk therapy halve the rate of cancer recurrence? How to think about the statistical significance of this finding? Is it just another example of the garden of forking paths?

James Coyne (who we last encountered in the sad story of Ellen Langer) writes: I’m writing to you now about another matter about which I hope you will offer an opinion. Here is a critique of a study, as well as the original study that claimed to find an effect of group psychotherapy on time […]

The connection between varying treatment effects and the well-known optimism of published research findings

Jacob Hartog writes: I thought this article [by Hunt Allcott and Sendhil Mullainathan], although already a couple of years old, fits very well into the themes of your blog—in particular the idea that the “true” treatment effect is likely to vary a lot depending on all kinds of factors that we can and cannot observe, […]

My talk at MIT this Thursday

When I was a student at MIT, there was no statistics department. I took a statistics course from Stephan Morgenthaler and liked it. (I’d already taken probability and stochastic processes back at the University of Maryland; my instructor in the latter class was Prof. Grace Yang, who was super-nice. I couldn’t follow half of what […]