Skip to content
Archive of posts filed under the Miscellaneous Statistics category.

Documented forking paths in the Competitive Reaction Time Task

Baruch Eitan writes: This is some luscious garden of forking paths. Indeed. Here’s what Malte Elson writes at the linked website: The Competitive Reaction Time Task, sometimes also called the Taylor Aggression Paradigm (TAP), is one of the most commonly used tests to purportedly measure aggressive behavior in a laboratory environment. . . . While […]

The p-value is a random variable

Sam Behseta sends along this paper by Laura Lazzeroni, Ying Lu, and Ilana Belitskaya-Lévy, who write: P values from identical experiments can differ greatly in a way that is surprising to many. The failure to appreciate this wide variability can lead researchers to expect, without adequate justification, that statistically significant findings will be replicated, only […]

A kangaroo, a feather, and a scale walk into Viktor Beekman’s office

E. J. writes: I enjoyed your kangaroo analogy [see also here—ed.] and so I contacted a talented graphical artist—Viktor Beekman—to draw it. The drawing is on Flickr under a CC license. Thanks, Viktor and E.J.!

What recommendations to give when a medical study is not definitive (which of course will happen all the time, especially considering that new treatments should be compared to best available alternatives, which implies that most improvements will be incremental at best)

Simon Gates writes: I thought you might be interested in a recently published clinical trial, for potential blog material. It picks up some themes that have cropped in recent months. Also, it is important for the way statistical methods influence what can be life or death decisions. The OPPTIMUM trial ( evaluated use of vaginal progesterone […]

Does Benadryl make you senile? Challenges in research communication

Mark Tuttle points to a post, “Common anticholinergic drugs like Benadryl linked to increased dementia risk” by Beverly Merz, Executive Editor, Harvard Women’s Health Watch. Merz writes: In a report published in JAMA Internal Medicine, researchers offers compelling evidence of a link between long-term use of anticholinergic medications like Benadryl and dementia. . . . […]

Call for research on California water resources

Patrick Atwater writes: I serve as a project manager of the California Data Collaborative, a coalition of water utilities working together to share data and ensure water reliability. We’ve put together a quick call for ideas on studies into the demand effects of water rates leveraging this unique database. California’s water world is highly fragmented […]

More evidence that even top researchers routinely misinterpret p-values

Blake McShane writes: I wanted to write to you about something related to your ongoing posts on replication in psychology as well as your recent post the ASA statement on p-values. In addition to the many problems you and others have documented with the p-value as a measure of evidence (both those computed “honestly” and […]

“Pointwise mutual information as test statistics”

Christian Bartels writes: Most of us will probably agree that making good decisions under uncertainty based on limited data is highly important but remains challenging. We have decision theory that provides a framework to reduce risks of decisions under uncertainty with typical frequentist test statistics being examples for controlling errors in absence of prior knowledge. […]

“Positive Results Are Better for Your Career”

Brad Stiritz writes: I thought you might enjoy reading the following Der Spiegel interview with Peter Wilmshurst. Talk about fighting the good fight! He took the path of greatest resistance, and he beat what I presume are pretty stiff odds. Then the company representatives asked me to leave some of the patients out of the […]

Data science as the application of theoretical knowledge

Patrick Atwater writes: Insights that “much of what’s hard looks easy” and it’s about “getting the damn data” highlight important points that much of the tech-ey industry dominating definitions overlook in the excitement about production ML recommendation systems and the like. Working to build from that grounded perspective, I penned together a quick piece digging […]

Hey—here’s a tip from the biology literature: If your correlation is .02, try binning your data to get a correlation of .8 or .9!

Josh Cherry writes: This isn’t in the social sciences, but it’s an egregious example of statistical malpractice: Below the abstract you can find my [Cherry’s] comment on the problem, which was submitted as a letter to the journal, but rejected on the grounds that the issue does not affect the main conclusions of the article […]

Racial classification sociology controversy update

The other day I posted on a controversy in sociology where Aliya Saperstein and Andrew Penner analyzed data from the National Longitudinal Survey of Youth, coming to the conclusion that “that race is not a fixed characteristic of individuals but is flexible and continually negotiated in everyday interactions,” but then Lance Hannon and Robert DeFina […]

“What is a good, convincing example in which p-values are useful?”

A correspondent writes: I came across this discussion of p-values, and I’d be very interested in your thoughts on it, especially on the evaluation in that thread of “two major arguments against the usefulness of the p-value:” 1. With large samples, significance tests pounce on tiny, unimportant departures from the null hypothesis. 2. Almost no […]

Taking responsibility for your statistical conclusions: You must decide what variation to compare to.

A couple people pointed me to a recent paper by Josh Terrell, Andrew Kofink, Justin Middleton, Clarissa Rainear, Emerson Murphy-Hill​, and Chris Parnin, “Gender bias in open source: Pull request acceptance of women versus men.” The term “bias” seems a bit loaded given the descriptive nature of their study. That said, it’s good for people […]

“The Natural Selection of Bad Science”

That’s the title of a new paper by Paul Smaldino and Richard McElreath which presents a sort of agent-based model that reproduces the growth in the publication of junk science that we’ve seen in recent decades. Even before looking at this paper I was positively disposed toward it for two reasons. First because I do […]

“Replication initiatives will not salvage the trustworthiness of psychology”

So says James Coyne, going full Meehl. I agree. Replication is great, but if you replicate noise studies, you’ll just get noise, hence the beneficial effects on science are (a) to reduce confidence in silly studies that we mostly shouldn’t have taken seriously in the first place, and (b) to provide an disincentive for future […]

All that really important statistics stuff that isn’t in the statistics textbooks

Kaiser writes: More on that work on age adjustment. I keep asking myself where is it in the Stats curriculum do we teach students this stuff? A class session focused on that analysis teaches students so much more about statistical thinking than anything we have in the textbooks. I’m not sure. This sort of analysis […]

Should he major in political science and minor in statistics or the other way around?

Andrew Wheeler writes: I will be a freshman at the University of Florida this upcoming fall and I am interested in becoming a political pollster. My original question was whether I should major in political science and minor in statistics or the other way around, but any other general advice would be appreciated. My reply: […]

“99.60% for women and 99.58% for men, P < 0.05.”

Gur Huberman pointed me to this paper by Tamar Kricheli-Katz and Tali Regev, “How many cents on the dollar? Women and men in product markets.” It appeared in something called ScienceAdvances, which seems to be some extension of the Science brand, i.e., it’s in the tabloids! I’ll leave the critical analysis of this paper to […]

Now that’s what I call a power pose!

John writes: See below for your humour file or blogging on a quiet day. . . . Perhaps you could start a competition for the wackiest real-life mangling of statistical concepts (restricted to a genuine academic setting?). On 15 Feb 2016, at 5:25 PM, [****] wrote: Pick of the bunch from tomorrow’s pile of applications […]