John writes:
See below for your humour file or blogging on a quiet day. . . . Perhaps you could start a competition for the wackiest real-life mangling of statistical concepts (restricted to a genuine academic setting?).
On 15 Feb 2016, at 5:25 PM, [****] wrote:
Pick of the bunch from tomorrow’s pile of applications at the [XYZ] Human Ethics Sub-Committee:
“. . . If the difference between the two scores is not as significant as predicted, a power calculation will be performed to determine if this is due to sampling . . .”
I love this. By which I mean, I hate this. Again, though, I have to pin much of the blame on the statistical profession, which has sold statistics to researchers as a way to distill 95% pure certainty out of randomness.
Statistics made a success out of the embodied-cognition guy, it made a hero out of the power-pose lady, and maybe it can do the same for you!
P.S. If you want something more to read today, there’s this that I posted on Retraction Watch. Some of the comments to that post were a bit . . . off. I think that lots of readers of that site have such strong views on retractions and corrections that they weren’t even reading what I wrote, they just teed off on what they thought I might be saying. It can be a challenge to write for this sort of audience.
Interesting that you blame the stats profession. Will you talk about this in your keynote at the M3 conference? Would be great if more stats people helped us fix things! (Some already are doing so, of course, just surprised that more aren’t.)
Simine:
I couldn’t remember what I’d be speaking on at this conference, so I checked my old emails. It looks like I’ll be giving the talk, “Easier said than done: Open problems in multilevel regression.” So it looks like I wasn’t planning to give my speech on how statistics is to blame for encouraging deterministic thinking. Maybe I can work it in somehow, though. Or you can ask a question on it at the end, if it doesn’t come up during the talk.
Sadly I won’t be at M3. But I would be very curious how a talk to statisticians about their role in the replicability situation would go over. I think there is a lot they can do to help, and it would be great to get more people interested.
Simon:
I have given the it’s-statisticians’-fault talk to statisticians and I don’t meet any resistance. I think the next step is to develop the new statistics curriculum that moves away from deterministic thinking and easy answers.
> next step is to develop the new statistics curriculum that moves away from deterministic thinking and easy answers
Agree and think part of that is the teaching and testing of formulas or even most methods to get solutions _are_ very deterministic.
A real need to get beyond the math but without being able to skip over much of the math…
Unfortunately, and surprisingly, Jacob Cohen, father of power analysis,misinterprets “power” in the first sentence of his famous book (and a fewpages in). See “Jacob Cohen’s slips” within my post today: https://errorstatistics.com/2016/05/22/frequentstein-whats-wrong-with-1-%CE%B2%CE%B1-as-a-measure-of-evidence-against-the-null/
To my knowledge, he computes power correctly throughout.
Drink every time someone says “p-value”.
Distill all you like, Andrew, but you’re going *against* the grain.
95 percent pure, distilled success