Someone sends me an email, “I don’t know what to do with this so I thought I would send it to you,” with a link to a university press release about a recently published research paper, full of silly statistical errors and signifying nothing.
I replied:
Can’t you just ignore this? Why give it any attention at all? There are millions of research papers published every year, right? Why play the publicists’ game? Let’s just leave this sort of thing for NPR, the Huffington Post, and other purveyors of junk science headlines.
You’ve been Kanazawa’ed
Daniel:
Kanazawa is the least of it. Back when I was reading Kanazawa’s papers . . . well, there was something logical about the whole situation: Kanazawa was an intentionally provocative writer with a knack for getting papers published even though (or especially because) they had ridiculous claims backed up by erroneous statistics. But it was easy to think of him as a special case.
The more recent world of psychology research is more disturbing because it’s not just headline-grabbing Kanazawas, it’s dozens (I guess hundreds, maybe thousands) of researchers all over the world using statistical errors to get headline-worthy claims.
We could laugh at Kanazawa because his claims were so ridiculous. It’s harder to laugh when it’s a whole subfield of psychology, backed up by Harvard professors and the like. Then it’s time to cry.
At least you don’t have too many friends:
https://www.washingtonpost.com/news/wonk/wp/2016/03/18/why-smart-people-are-better-off-with-fewer-friends/
When I saw this a few days ago I figured people would be sending it to you like crazy, so naturally I assumed :-)
Oh, no, I didn’t hear about that. How horrible that a reporter at a reputable newspaper would take this guy’s research seriously. Not that it’s necessarily wrong, but given his track record, I’d recommend some skepticism!
“Wonkblog,” indeed.
> “Wonkblog,” indeed.
I think they’re spelling it wrong. Based on site content I say the first “o” should be an “a”.
It’s funny how the Washington Post blocks access to online readers, demanding payment to read the article. What century are these people living in? I will just not read the article; did they consider that that’s probably going to be the most common response?
strange because I can click it and get the article no problem.
Are you on a university campus that might subscribe to the WAPO? That might explain it. When I am off campus I have to use a proxy server to get things that I can get on campus just by clicking on them.
nope, just at home on a regular old cable internet connection.
Yes. That one’s an annoying problem. My pet solution was setting up on my laptop a Putty ssh login session to a linux machine at work. In Putty enabled port forwarding and set up a ssh tunnel with a SOCKS proxy & then install a local chrome plugin that would route only selected domains to the tunnel.
Each time I couldn’t read a paywalled article click the plugin to reroute all future visits to that site / journal / newspaper.
Worked like a charm.
> maybe thousands) of researchers all over the world using statistical errors to get headline-worthy claims
Perhaps makes it easy to forget some folks do careful self-critical work and it is possible to do good work (though likely you will have to work harder for less recognition).
Some talk on the radio this morning that some researchers are discerning that the apparent health benefits of moderate drinking were/are due to selection confounding in the non-randomized studies…
Now there would be a risk of carefully looking into methodology they brought to bear on that ;-)
Politics around drinking are almost as bad as politics around guns. Remember this one: http://statmodeling.stat.columbia.edu/2015/09/30/an-unconvincing-analysis-claiming-to-debunk-the-health-benefits-of-moderate-drinking/
The thing is, it’s possible to do good work… for a while, until the money dries up because you aren’t as “productive” as your “peers” who publish all this other crap.
Ironic Stockholmed Stockholm posting is ironic.
http://statmodeling.stat.columbia.edu/2016/02/23/an-apology-and-a-note-on-stockholm-syndrome/
Jrc:
Oh, that one makes me sad, still! Poor guy.
Gee, you forgot the obligatory xkcd cartoon :)
https://xkcd.com/386/
Out, out, brief study!
It’s but a printed shadow, a poor paper,
That struts and frets its hour upon the page,
And then is seen no more. It is a tale
Told by an idiot, full of silly stats errors,
Signifying nothing.
Hey, who do you think you are? John Kruschke? ;)
Andrew
I’ve occasionally sent you an few email like that. Would you rather I didn’t?
Ethan:
No, I like those emails. It’s just this particular one sent me over the edge.
hahahahahahaha
so happy that ain’t me.
i’ll continue indulging in the perceived ‘village idiot’ persona.
my phone rings, MAYBE, a few times a month.
my inbox receives, MAYBE, one or two “work” emails a month.
:))))))))
Can you recommend 2 or 3 questions that people at these news websites could ask, to use as a filter before deciding what to cover?
Not sure about Andrew, but usually what I care about is predictions, independent replications, and comparing various explanations. So some questions about:
1) What previously proposed predictions has your study ruled out or not?
2) What new predictions have your results lead you to make that we should see tested in the future?
3) Did your work replicate the results, in whole or in part, those of any previous research?
4) If there are plans for anyone to attempt replicating the current study, what difficulties may they face?
5) What are alternative possible explanations for your results, how difficult will it be to distinguish between these in the future?
The language may need to be spiced up to give it any wide appeal…