The importance of measurement, and how you can draw ridiculous conclusions from your statistical analyses if you don’t think carefully about measurement . . . Leamer (1983) got it.

Jacob Klerman writes: I have noted your recent emphasis on the importance of measurement (e.g., “Here are some ways to make your study replicable…”). For reasons not relevant here, I was rereading Leamer (1983), Let’s Take the Con Out of … Continue reading

“Not within spitting distance”: Challenges in measuring ovulation, as an example of the general issue of the importance of measurement in statistics

Ruben Arslan writes: I don’t know how interested you still are in what’s going on in ovulation research, but I hoped you might find the attached piece interesting. Basically, after the brouhaha following Harris et al. 2013 observation that the … Continue reading

How to reconcile that I hate structural equation models, but I love measurement error models and multilevel regressions, even though these are special cases of structural equation models?

Andy Dorsey writes: I’m a graduate student in psychology. I’m trying to figure out what seems to me to be a paradox: One issue you’ve talked about in the past is how you don’t like structural equation modeling (e.g., your … Continue reading

It’s kinda like phrenology but worse. Not so good for the “Nature” brand name, huh? Measurement, baby, measurement.

Federico Mattiello writes: I thought you might find this thread interesting, it’s about a machine learning paper building a “trustworthiness score” from faces databases and historical (mainly British) portraits. It checks many bias boxes I believe, but my biggest complaint … Continue reading

Pizzagate: The problem’s not with the multiple analyses, it’s with the selective reporting of results (and with low-quality measurements and lack of quality control all over, but that’s not the key part of the story)

“I don’t think I’ve ever done an interesting study where the data ‘came out’ the first time I looked at it.” — Brian Wansink The funny thing is, I don’t think this quote is so bad. Nothing comes out right … Continue reading

It’s not enough to be a good person and to be conscientious. You also need good measurement. Cargo-cult science done very conscientiously doesn’t become good science, it just falls apart from its own contradictions.

Kevin Lewis points us to a biology/psychology paper that was a mix of reasonable null claims (on the order of, the data don’t give us enough information to say anything about XYZ) and some highly questionable noise mining supported by … Continue reading