Skip to content
Archive of entries posted by

“She also observed that results from smaller studies conducted by NGOs – often pilot studies – would often look promising. But when governments tried to implement scaled-up versions of those programs, their performance would drop considerably.”

Robert Wiblin writes: If we have a study on the impact of a social program in a particular place and time, how confident can we be that we’ll get a similar result if we study the same program again somewhere else? Dr Eva Vivalt . . . compiled a huge database of impact evaluations in […]

A Bayesian take on ballot order effects

Dale Lehman sends along a paper, “The ballot order effect is huge: Evidence from Texas,” by Darren Grant, which begins: Texas primary and runoff elections provide an ideal test of the ballot order hypothesis, because ballot order is randomized within each county and there are many counties and contests to analyze. Doing so for all […]

“The hype economy”

Palko writes: I have no idea whether it is real or apocryphal, but there’s an often referred to study with primates where the they earned tokens that could be exchanged for food. According to the standard version, the subjects soon came to value those tokens more than the treats they could be exchanged for. The […]

Tom Wolfe

I’m a big Tom Wolfe fan. My favorites are The Painted Word and From Bauhaus to Our House, and I have no patience for the boosters (oh, sorry, “experts”) of modern art of the all-black-painting variety or modern architecture of the can’t-find-the-front-door variety who can’t handle Wolfe’s criticism. I also enjoyed Bonfire of the Vanities, […]

Graphs and tables, tables and graphs

Jesse Wolfhagen writes: I was surprised to see a reference to you in a Quartz opinion piece entitled “Stop making charts when a table is better”. While the piece itself makes that case that there are many kinds of charts that are simply restatements of tabular data, I was surprised that you came up as […]

“Using numbers to replace judgment”

Julian Marewski and Lutz Bornmann write: In science and beyond, numbers are omnipresent when it comes to justifying different kinds of judgments. Which scientific author, hiring committee-member, or advisory board panelist has not been confronted with page-long “publication manuals”, “assessment reports”, “evaluation guidelines”, calling for p-values, citation rates, h-indices, or other statistics in order to […]

2018: How did people actually vote? (The real story, not the exit polls.)

Following up on the post that we linked to last week, here’s Yair’s analysis, using Mister P, of how everyone voted. Like Yair, I think these results are much better than what you’ll see from exit polls, partly because the analysis is more sophisticated (MRP gives you state-by-state estimates in each demographic group), partly because […]

Hey, check this out: Columbia’s Data Science Institute is hiring research scientists and postdocs!

Here’s the official announcement: The Institute’s Postdoctoral and Research Scientists will help anchor Columbia’s presence as a leader in data-science research and applications and serve as resident experts in fostering collaborations with the world-class faculty across all schools at Columbia University. They will also help guide, plan and execute data-science research, applications and technological innovations […]

The State of the Art

Christie Aschwanden writes: Not sure you will remember, but last fall at our panel at the World Conference of Science Journalists I talked with you and Kristin Sainani about some unconventional statistical methods being used in sports science. I’d been collecting material for a story, and after the meeting I sent the papers to Kristin. […]

Robustness checks are a joke

Someone pointed to this post from a couple years ago by Uri Simonsohn, who correctly wrote: Robustness checks involve reporting alternative specifications that test the same hypothesis. Because the problem is with the hypothesis, the problem is not addressed with robustness checks. Simonsohn followed up with an amusing story: To demonstrate the problem I [Simonsohn] […]

Chocolate milk! Another stunning discovery from an experiment on 24 people!

Mike Hull writes: I was reading over this JAMA Brief Report and could not figure out what they were doing with the composite score. Here are the cliff notes: Study tested milk vs dark chocolate consumption on three eyesight performance parameters: (1) High-contrast visual acuity (2) Small-letter contrast sensitivity (3) Large-letter contrast sensitivity Only small-letter […]

“Law professor Alan Dershowitz’s new book claims that political differences have lately been criminalized in the United States. He has it wrong. Instead, the orderly enforcement of the law has, ludicrously, been framed as political.”

This op-ed by Virginia Heffernan is about g=politics, but it reminded me of the politics of science. Heffernan starts with the background: This last year has been a crash course in startlingly brutal abuses of power. For decades, it seems, a caste of self-styled overmen has felt liberated to commit misdeeds with impunity: ethical, sexual, […]

Hey! Here’s what to do when you have two or more surveys on the same population!

This problem comes up a lot: We have multiple surveys of the same population and we want a single inference. The usual approach, applied carefully by news organizations such as Real Clear Politics and Five Thirty Eight, and applied sloppily by various attention-seeking pundits every two or four years, is “poll aggregation”: you take the […]

2018: Who actually voted? (The real story, not the exit polls.)

Continuing from our earlier discussion . . . Yair posted some results from his MRP analysis of voter turnout: 1. The 2018 electorate was younger than in 2014, though not as young as exit polls suggest. 2. The 2018 electorate was also more diverse, with African American and Latinx communities surpassing their share of votes […]

Matching (and discarding non-matches) to deal with lack of complete overlap, then regression to adjust for imbalance between treatment and control groups

John Spivack writes: I am contacting you on behalf of the biostatistics journal club at our institution, the Mount Sinai School of Medicine. We are working Ph.D. biostatisticians and would like the opinion of a true expert on several questions having to do with observational studies—questions that we have not found to be well addressed […]

2018: What really happened?

We’re always discussing election results on three levels: their direct political consequences, their implications for future politics, and what we can infer about public opinion. In 2018 the Democrats broadened their geographic base, as we can see in this graph from Yair Ghitza: Party balancing At the national level, what happened is what we expected […]

“Recapping the recent plagiarism scandal”

Benjamin Carlisle writes: A year ago, I received a message from Anna Powell-Smith about a research paper written by two doctors from Cambridge University that was a mirror image of a post I wrote on my personal blog roughly two years prior. The structure of the document was the same, as was the rationale, the […]

Watch out for naively (because implicitly based on flat-prior) Bayesian statements based on classical confidence intervals! (Comptroller of the Currency edition)

Laurent Belsie writes: An economist formerly with the Consumer Financial Protection Bureau wrote a paper on whether a move away from forced arbitration would cost credit card companies money. He found that the results are statistically insignificant at the 95 percent (and 90 percent) confidence level. But the Office of the Comptroller of the Currency […]

“35. What differentiates solitary confinement, county jail and house arrest” and 70 others

Thomas Perneger points us to this amusing quiz on statistics terminology: Lots more where that came from.

“Statistical and Machine Learning forecasting methods: Concerns and ways forward”

Roy Mendelssohn points us to this paper by Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos, which begins: Machine Learning (ML) methods have been proposed in the academic literature as alternatives to statistical ones for time series forecasting. Yet, scant evidence is available about their relative performance in terms of accuracy and computational requirements. The purpose […]