Hector Cordero-Guzman writes:
I have a question for you about an ongoing congroversy\incident related to reporting of social science research.
Please see article linked below if you have a chance. I think this incident exposes real problems in the way social science research is presented and how it reaches the public…
Essentially, we have a story in the NY Times based on a blogpost of a presentation made at a professional conference (that requires papers) that no one has seen and on a study no outsider has apparently reviewed. I asked the authors for the paper and they refused to share it. They did say they could talk to me on the phone about it…
Even after all of the concerns that were raised in the link below (including to the editor of the story at the NYT) and in other forums about the study\report not being public or available for outside academic experts and the authors not wanting to share it with me or others in the press to review, the New York Times went ahead and put this on Twitter on SUNDAY (see below)!!! To their over 11 million followers!!!
I told the authors that the way this has been handled has been very poor and is not and should not be best practice in terms of how scientific research is reviewed, released to the public, covered in blogs and the media, or discussed.
If they presented work that was not for discussion or attribution (at an academic conference that requires papers for presentations, BTW) then Pew should not have written about it and that should have been very clear to all…If they presented work at an academic conference at least their presentation should be immediately available for others to review. The more potentially damaging or stigmatizing or controversial the work the more authors have to be careful how they put it out in public.
The process broke down and my sense is that the research team (including, potentially, co-authors that work at the Census Bureau) has some responsibility for that.
Peer review and transparency are essential for science. If authors are presenting something, they should take responsibility for it and let others see the work. If it is not ready then it should not be presented or it should be discussed at a faculty seminar or workshop with clear ground rules…
The process was either not followed or broke down in this instance. Neither is good.
[And, I want to be clear that I have no objection to the study or the findings—whatever they are. I have only seen what is reported but have no way of judging accuracy, veracity, validity, reliability or anything else and that—to me—is the biggest problem right now…]
Do you agree? Is this the way to present social science research to the public?
I do not want to make a bigger public issue about this at the moment (though I have been quite vocal on twitter and facebook about this) but would realy appreciate your expert assesment, opinion, and any suggestions you may have.
I don’t want to feel that I am going after coleagues unfairly or making an issue where one does not really exist.
I have mixed feelings about this sort of story. On one hand I agree with him that there are problems when scientific communication is conducted via press release rather than peer review. And it’s not good when people won’t share the articles on which their conclusions are based. On the other hand, if the topic is important you can’t really wait until the article is accepted for publication. In some fields (such as statistics and political science) the publication process can be slow. Also, as we keep discussing, there’s nothing so special about publication. All that publication does is to fix a paper in some permanent form; it doesn’t mean the result is correct or even reasonable.