Can’t those shameless little bullies just let scientists do their research in peace?
If a hypothesis test is statistically significant and a result is published in a real journal, that should be enough for any self-styled skeptic.
Can you imagine what might happen if any published result could be questioned—by anybody? You’d have serious psychology research being grilled by statisticians, and biology research being called into question by . . . political scientists?
Where did that come from? I don’t ask my hairdresser to check my math calculations, I don’t ask my house cleaner to repair my TV, and I sure as heck wouldn’t trust a political scientist to vet a biology paper.
When something’s published in the Journal of Theoretical Biology, pal, it’s theoretical biology. It’s not baseball statistics. Don’t send an amateur to do a pro’s job.
Simply put, peer review is a method by which scientists who are experts in a particular field examine another scientist’s work to verify that it makes a valid contribution to the evidence base. With that assurance, a scientist can report his or her work to the public, and the public can trust the work.
And then there’s multiple comparisons. Or should I say, “p hacking.” Christ, what a load of bull. You replication twits are such idiots. If I publish a paper with 9 statistically significant results, do you know what the probability of this is, if the null hypothesis were true? It’s (1/20) to the 9th power. Actually much lower than that: you’d get (1/20)^9 if all the p-values were .05, but actually some of these p-values are even lower, like .01 or .001. Anyway, even if it’s just (1/20)^9, do you know how low that is?
Probably not, you innumerate git.
So let me tell you, it’s 1.953125e-12, that’s 0.00000000000195312. Got that? No way any amount of multiple comparisons can cover that one. If I find 9 statistically significant results, my result is real. Period. I don’t care how many people can’t replicate it. If they can’t replicate it, it’s their problem.
p < 0.00000000000195312. You can take that one to the bank, pal.
OK, let’s be systematic. Suppose I do a study and it is statistically significant and I publish it—it’s hard to get a paper published, dontcha know?—and then some little Dutch second-stringers raise some pissant objection on some blog, and then they sandbag me with some lame-a$$ “replication.” OK, fine. There are two possibilities, then:
1. My study replicates. Good. So shut the f^&#!@ up. Or,
2. The so-called replication fails. This doesn’t mean squat. All it tells us is that the world is complicated. We already knew that.
Science is about exploration, not criticism. Let’s be open-minded. Personally, I’m open-minded enough to believe that women’s political preferences change by 20 percentage points during their monthly cycle. Why not? What are you, anti-science? OK, ok, I’m not so sure that Daryl Bem found ESP—but I think we’re a damn sight better off giving him the benefit of the doubt, than censoring any result that doesn’t fit our high-and-mighty idea of what is proper science.
Jean Piaget never did a preregistered replication. Nor, for that matter, did B. F. Skinner or Sigmund Freud or Barbara Fredrickson or that Dianetics guy or all the other leading psychologists of the last two centuries.
What did Piaget and the rest of those guys did? They did what all the best scientists did: they ran questionnaires on Mechanical Turk, they found p<.05, and they published in Psychological Science. If it was good enough for Jean Piaget and B. F. Skinner and William James and Daryl Bem and Satoshi Kanazawa, it's good enough for me.
So take those replications and stick ‘em where the sun don’t shine, then crawl back under the rock where you came from, you little twerp. The rest of us won’t even notice. Why? Cos, while you’re sniping and criticizing and replicating and blogging, we’re busy in our labs. Doing science.