Skip to content
Archive of entries posted by

Post-publication review succeeds again! (Two-lines edition.)

A couple months ago, Uri Simonsohn posted online a suggested statistical method for detecting nonmonotonicity in data. He called it: “Two-lines: The First Valid Test of U-Shaped Relationships.” With a title like that, I guess you’re asking for it. And, indeed, awhile later I received an email from Yair Heller identifying some problems with Uri’s […]

Pseudoscience and the left/right whiplash

I came across this post by blogger Echidne slamming psychology professor Roy Baumeister. I’d first heard about the Baumeister in the context of his seeming inability to handle scientific criticism. I hadn’t realized that Baumeister had a sideline in pseudoscientific anti-political-correctness. One aspect of all this that interests me is the way that Baumeister, and […]

More thoughts on that “What percent of Americans would you say are gay or lesbian?” survey

We had some discussion yesterday about this Gallup poll that asked respondents to guess the percentage of Americans who are gay. The average response was 23%—and this stunningly high number was not just driven by outliers: more than half the respondents estimated the proportion gay as 20% or more. All this is in stark contrast […]

Looking for data on speed and traffic accidents—and other examples of data that can be fit by nonlinear models

[cat picture] For the chapter in Regression and Other Stories that includes nonlinear regression, I’d like a couple homework problems where the kids have to construct and fit models to real data. So I need some examples. We already have the success of golf putts as a function of distance from the hole, and I’d […]

Statistical Significance and the Dichotomization of Evidence (McShane and Gal’s paper, with discussions by Berry, Briggs, Gelman and Carlin, and Laber and Shedden)

Blake McShane sent along this paper by himself and David Gal, which begins: In light of recent concerns about reproducibility and replicability, the ASA issued a Statement on Statistical Significance and p-values aimed at those who are not primarily statisticians. While the ASA Statement notes that statistical significance and p-values are “commonly misused and misinterpreted,” […]

“Americans Greatly Overestimate Percent Gay, Lesbian in U.S.”

This sort of thing is not new but it’s still amusing. From a Gallup report by Frank Newport: The American public estimates on average that 23% of Americans are gay or lesbian, little changed from Americans’ 25% estimate in 2011, and only slightly higher than separate 2002 estimates of the gay and lesbian population. These […]

Using Mister P to get population estimates from respondent driven sampling

From one of our exams: A researcher at Columbia University’s School of Social Work wanted to estimate the prevalence of drug abuse problems among American Indians (Native Americans) living in New York City. From the Census, it was estimated that about 30,000 Indians live in the city, and the researcher had a budget to interview […]


Kevin Lewis points to a research article by Lawton Swan, John Chambers, Martin Heesacker, and Sondre Nero, “How should we measure Americans’ perceptions of socio-economic mobility,” which reports effects of question wording on surveys on an important topic in economics. They replicated two studies: Each (independent) research team had prompted similar groups of respondents to […]

“Quality control” (rather than “hypothesis testing” or “inference” or “discovery”) as a better metaphor for the statistical processes of science

I’ve been thinking for awhile that the default ways in which statisticians think about science—and which scientists think about statistics—are seriously flawed, sometimes even crippling scientific inquiry in some subfields, in the way that bad philosophy can do. Here’s what I think are some of the default modes of thought: – Hypothesis testing, in which […]

Advice for science writers!

I spoke today at a meeting of science journalists, in a session organized by Betsy Mason, also featuring Kristin Sainani, Christie Aschwanden, and Tom Siegfried. My talk was on statistical paradoxes of science and science journalism, and I mentioned the Ted Talk paradox, Who watches the watchmen, the Eureka bias, the “What does not kill […]

My favorite definition of statistical significance

From my 2009 paper with Weakliem: Throughout, we use the term statistically significant in the conventional way, to mean that an estimate is at least two standard errors away from some “null hypothesis” or prespecified value that would indicate no effect present. An estimate is statistically insignificant if the observed value could reasonably be explained […]

An alternative to the superplot

Kevin Brown writes: I came across the lexicon link to your ‘super plots’ posting today. In it, you plot the association between individual income (X) and republican voting (Y) for 3 states: one assumed to be poor, one middle income, and one wealthy. An alternative way of plotting this, what I call a ‘herd effects […]

Science funding and political ideology

Mark Palko points to this news article by Jeffrey Mervis entitled, “Rand Paul takes a poke at U.S. peer-review panels”: Paul made his case for the bill yesterday as chairperson of a Senate panel with oversight over federal spending. The hearing, titled “Broken Beakers: Federal Support for Research,” was a platform for Paul’s claim that […]

Quick Money

I happened to come across this Los Angeles Times article from last year: Labor and business leaders declared victory Tuesday night over a bitterly contested ballot measure that would have imposed new restrictions on building apartment towers, shops and offices in Los Angeles. As of midnight, returns showed Measure S going down to defeat by […]

If you want to know about basketball, who ya gonna trust, a mountain of p-values . . . or that poseur Phil Jackson??

Someone points me with amusement to this published article from 2012: Beliefs About the “Hot Hand” in Basketball Across the Adult Life Span Alan Castel, Aimee Drolet Rossi, and Shannon McGillivray University of California, Los Angeles Many people believe in streaks. In basketball, belief in the “hot hand” occurs when people think a player is […]

In the open-source software world, bug reports are welcome. In the science publication world, bug reports are resisted, opposed, buried.

Mark Tuttle writes: If/when the spirit moves you, you should contrast the success of the open software movement with the challenge of published research. In the former case, discovery of bugs, or of better ways of doing things, is almost always WELCOMED. In some cases, submitters of bug reports, patches, suggestions, etc. get “merit badges” […]

This Friday at noon, join this online colloquium on replication and reproducibility, featuring experts in economics, statistics, and psychology!

Justin Esarey writes: This Friday, October 27th at noon Eastern time, the International Methods Colloquium will host a roundtable discussion on the reproducibility crisis in social sciences and a recent proposal to impose a stricter threshold for statistical significance. The discussion is motivated by a paper, “Redefine statistical significance,” recently published in Nature Human Behavior (and available […]

I think it’s great to have your work criticized by strangers online.

Brian Resnick writes: I’m hoping you could help me out with a story I’m looking into. I’ve been reading about the debate over how past work should be criticized and in what forums. (I’m thinking of the Susan Fiske op-ed against using social media to “bully” authors of papers that are not replicating. But […]

My 2 talks in Seattle this Wed and Thurs: “The Statistical Crisis in Science” and “Bayesian Workflow”

For the Data Science Seminar, Wed 25 Oct, 3:30pm in Physics and Astronomy Auditorium – A102: The Statistical Crisis in Science Top journals routinely publish ridiculous, scientifically implausible claims, justified based on “p < 0.05.” And this in turn calls into question all sorts of more plausible, but not necessarily true, claims, that are supported […]

The Publicity Factory: How even serious research gets exaggerated by the process of scientific publication and reporting

The starting point is that we’ve seen a lot of talk about frivolous science, headline-bait such as the study that said that married women are more likely to vote for Mitt Romney when ovulating, or the study that said that girl-named hurricanes are more deadly than boy-named hurricanes, and at this point some of these […]