“One of the things that makes scientific research hard is that one is usually not sure what hat one should be wearing in the given situation.” — David H. Krantz, 1938-2023 It’s been a bunch of years since I talked … Continue reading
I happened to come across this cool article from 2001 by Daniel Robinson and Howard Wainer. It was published in the Journal of Wildlife Management, of all places—I didn’t know that Howard ever worked in that area!—but you can find … Continue reading
This is Jessica. As you might expect, as a professor in a computer science department I spend a lot of time around computer scientists. As someone who is probably more outward looking than the average faculty member, there are things … Continue reading
This is Jessica. Andrew recently blogged in response to an article by McDermott arguing that pre-registration has costs like being unfair to junior scholars. I agree with his view that pre-registration can be a pre-condition for good science but not … Continue reading
This is Jessica. In a paper to appear at AIES 2022, Sayash Kapoor, Priyanka Nanayakkara, Arvind Narayanan, and Andrew and I write: Recent arguments that machine learning (ML) is facing a reproducibility and replication crisis suggest that some published claims … Continue reading
Michael Nelson writes: In a comment on your Saturday post, you linked to a previous post on Brandolini’s law, which I followed and read. I was especially struck by the post script, where you quote a commenter who gave a … Continue reading
This is Jessica. I’ve subscribed to aspects of the “estimation” movement–the move toward emphasizing magnitude and uncertainty of effects and testing multiple hypotheses rather than NHST–for awhile, having read this blog for years and switched over to using Bayesian stats … Continue reading
Allan Cousins (commenter AllanC on the blog) writes: Just wanted to let you know that at least 3 other of my fellow students at Sheffield are taking the graduate certificate because they noticed Shravan’s comments about the program on the … Continue reading
Javier Benítez pointed me to this JAMA article by Paul Young, Christopher Nickson, and Anders Perner, “When Should Clinicians Act on Non–Statistically Significant Results From Clinical Trials?”, which begins: Understanding whether the results of a randomized clinical trial (RCT) are … Continue reading
This is Jessica. Earlier this year, I was reading some of the emerging work on AI bias and fairness, and it got me thinking about the nature of claims (and common concerns about them) in AI/ML research compared to those … Continue reading
Evan Warfel asks a question: Let’s say that a researcher is collecting data on people for an experiment. Furthermore, it just so happens that due to the data collection procedure, data is gathered and recorded in 100-person increments. (Making it … Continue reading
Jonathan Falk points to this post by Richard Morey, who writes: I [Morey] am convinced that most experienced scientists and statisticians have internalized statistical insights that frequentist statistics attempts to formalize: how you can be fooled by randomness; how what … Continue reading
This is Jessica. I pay some attention to what gets discussed in methodological/statistical reform research and discussions, and I’m probably not the only one who’s watched as the movement (at least in psychology) seems to be getting more self-aware recently. … Continue reading
So I left in mid-lecture tempted by a reform song The plenary hall it shifted as they turned to watch me leave And I pulled a little p-curve from the pocket in my sleeve The variation it was stronger to … Continue reading
Jonathan Falk writes: As you have tirelessly promoted, a huge problem with NHST is that “insignificant” effects on average can mask, via attenuation bias, important changes in subgroups. Further, as you have somewhat less tirelessly pointed out, you need much … Continue reading
Richard Born writes: The practice of arbitrarily thresholding p values is not only deeply embedded in statistical practice, it is also congenial to the human mind. It is thus not sufficient to tell our students, “Don’t do this.” We must … Continue reading
Bill Harris writes: Thanks for posting my question the other day. Here’s another, somewhat related question. What if “your side” wins? What if, starting today, every analysis is done properly? Null hypothesis significance testing is something you read about only … Continue reading
The philosopher wrote: The big move in the statistics wars these days is to fight irreplication by making it harder to reject, and find evidence against, a null hypothesis. Mayo is referring to, among other things, the proposal to “redefine … Continue reading
Oliver Schultheiss writes: I am a regular reader of your blog. I am also one of those psychology researchers who were trained in the NHST tradition and who is now struggling hard to retrain himself to properly understand and use … Continue reading
The reason for log transforming your data is not to deal with skewness or to get closer to a normal distribution; that’s rarely what we care about. Validity, additivity, and linearity are typically much more important. The reason for log … Continue reading