Skip to content
Archive of posts filed under the Miscellaneous Statistics category.

Publication bias occurs within as well as between projects

Kent Holsinger points to this post by Kevin Drum entitled, “Publication Bias Is Boring. You Should Care About It Anyway,” and writes: I am an evolutionary biologist, not a psychologist, but this article describes a disturbing Scenario concerning oxytocin research that seems plausible. It is also relevant to the reproducibility/publishing issues you have been discussing […]

Better to just not see the sausage get made

Mike Carniello writes: This article in the NYT leads to the full text, in which these statement are buried (no pun intended): What is the probability that two given texts were written by the same author? This was achieved by posing an alternative null hypothesis H0 (“both texts were written by the same author”) and […]

A day in the life

I like to post approx one item per day on this blog, so when multiple things come up in the same day, I worry about the sustainability of all this. I suppose I could up the posting rate to 2 a day but I think that could be too much of a burden on the […]

One more thing you don’t have to worry about

Baruch Eitam writes: So I have been convinced by the futility of NHT for my scientific goals and by the futility of of significance testing (in the sense of using p-values as a measure of the strength of evidence against the null). So convinced that I have been teaching this for the last 2 years. […]

Kaiser Fung on the ethics of data analysis

Kaiser gave a presentation and he’s sharing the slides with us here. It’s important stuff.

The history of characterizing groups of people by their averages

Andrea Panizza writes: I stumbled across this article on the End of Average. I didn’t know about Todd Rose, thus I had a look at his Wikipedia entry: Rose is a leading figure in the science of individual, an interdisciplinary field that draws upon new scientific and mathematical findings that demonstrate that it is not […]

Will youths who swill Red Bull become adult cocaine addicts?

The above is the question asked to me by Michael Stutzer, who writes: I have attached an increasingly influential paper [“Effects of Adolescent Caffeine Consumption on Cocaine Sensitivity,” by Casey O’Neill, Sophia Levis, Drew Schreiner, Jose Amat, Steven Maier, and Ryan Bachtell] purporting to show the effects of caffeine use in adolescents (well, lab rats […]

Documented forking paths in the Competitive Reaction Time Task

Baruch Eitan writes: This is some luscious garden of forking paths. Indeed. Here’s what Malte Elson writes at the linked website: The Competitive Reaction Time Task, sometimes also called the Taylor Aggression Paradigm (TAP), is one of the most commonly used tests to purportedly measure aggressive behavior in a laboratory environment. . . . While […]

The p-value is a random variable

Sam Behseta sends along this paper by Laura Lazzeroni, Ying Lu, and Ilana Belitskaya-Lévy, who write: P values from identical experiments can differ greatly in a way that is surprising to many. The failure to appreciate this wide variability can lead researchers to expect, without adequate justification, that statistically significant findings will be replicated, only […]

A kangaroo, a feather, and a scale walk into Viktor Beekman’s office

E. J. writes: I enjoyed your kangaroo analogy [see also here—ed.] and so I contacted a talented graphical artist—Viktor Beekman—to draw it. The drawing is on Flickr under a CC license. Thanks, Viktor and E.J.!

What recommendations to give when a medical study is not definitive (which of course will happen all the time, especially considering that new treatments should be compared to best available alternatives, which implies that most improvements will be incremental at best)

Simon Gates writes: I thought you might be interested in a recently published clinical trial, for potential blog material. It picks up some themes that have cropped in recent months. Also, it is important for the way statistical methods influence what can be life or death decisions. The OPPTIMUM trial ( evaluated use of vaginal progesterone […]

Does Benadryl make you senile? Challenges in research communication

Mark Tuttle points to a post, “Common anticholinergic drugs like Benadryl linked to increased dementia risk” by Beverly Merz, Executive Editor, Harvard Women’s Health Watch. Merz writes: In a report published in JAMA Internal Medicine, researchers offers compelling evidence of a link between long-term use of anticholinergic medications like Benadryl and dementia. . . . […]

Call for research on California water resources

Patrick Atwater writes: I serve as a project manager of the California Data Collaborative, a coalition of water utilities working together to share data and ensure water reliability. We’ve put together a quick call for ideas on studies into the demand effects of water rates leveraging this unique database. California’s water world is highly fragmented […]

More evidence that even top researchers routinely misinterpret p-values

Blake McShane writes: I wanted to write to you about something related to your ongoing posts on replication in psychology as well as your recent post the ASA statement on p-values. In addition to the many problems you and others have documented with the p-value as a measure of evidence (both those computed “honestly” and […]

“Pointwise mutual information as test statistics”

Christian Bartels writes: Most of us will probably agree that making good decisions under uncertainty based on limited data is highly important but remains challenging. We have decision theory that provides a framework to reduce risks of decisions under uncertainty with typical frequentist test statistics being examples for controlling errors in absence of prior knowledge. […]

“Positive Results Are Better for Your Career”

Brad Stiritz writes: I thought you might enjoy reading the following Der Spiegel interview with Peter Wilmshurst. Talk about fighting the good fight! He took the path of greatest resistance, and he beat what I presume are pretty stiff odds. Then the company representatives asked me to leave some of the patients out of the […]

Data science as the application of theoretical knowledge

Patrick Atwater writes: Insights that “much of what’s hard looks easy” and it’s about “getting the damn data” highlight important points that much of the tech-ey industry dominating definitions overlook in the excitement about production ML recommendation systems and the like. Working to build from that grounded perspective, I penned together a quick piece digging […]

Hey—here’s a tip from the biology literature: If your correlation is .02, try binning your data to get a correlation of .8 or .9!

Josh Cherry writes: This isn’t in the social sciences, but it’s an egregious example of statistical malpractice: Below the abstract you can find my [Cherry’s] comment on the problem, which was submitted as a letter to the journal, but rejected on the grounds that the issue does not affect the main conclusions of the article […]

Racial classification sociology controversy update

The other day I posted on a controversy in sociology where Aliya Saperstein and Andrew Penner analyzed data from the National Longitudinal Survey of Youth, coming to the conclusion that “that race is not a fixed characteristic of individuals but is flexible and continually negotiated in everyday interactions,” but then Lance Hannon and Robert DeFina […]

“What is a good, convincing example in which p-values are useful?”

A correspondent writes: I came across this discussion of p-values, and I’d be very interested in your thoughts on it, especially on the evaluation in that thread of “two major arguments against the usefulness of the p-value:” 1. With large samples, significance tests pounce on tiny, unimportant departures from the null hypothesis. 2. Almost no […]