This story is pretty horrifying/funny. But the strangest thing was this part:
[The author] and her colleague have appealed to the unnamed journal, which belongs to the PLoS family . . .
I thought PLOS published just about everything! This is not a slam on PLOS. Arxiv publishes everything too, and Arxiv is great.
The funny thing is, I do think there are cases where having both male and female coauthors gives a paper more credibility, sometimes undeserved. For example, if you take a look at those papers on ovulation and voting, and ovulation and clothing, and fat arms and political attitudes, you’ll see these papers have authors of both sexes, which insulates them from the immediate laugh-them-out-of-the-room reaction that they might get were they written by men only. Having authors of both sexes does not of course exempt them from direct criticisms of the work; I just think that a paper on “that time of the month” written by men would, for better or worse, get a more careful review.
P.S. Also, one thing I missed in my first read of this story: the reviewer wrote:
Perhaps it is not so surprising that on average male doctoral students co-author one more paper than female doctoral students, just as, on average, male doctoral students can probably run a mile race a bit faster than female doctoral students . . . And it might well be that on average men publish in better journals . . . perhaps simply because men, perhaps, on average work more hours per week than women, due to marginally better health and stamina.
“Marginally better health and stamina”—that’s a laff and a half! Obviously this reviewer is no actuary and doesn’t realize that men die at a higher rate than women at every age.
On the plus side, it’s pretty cool that James Watson is still reviewing journal articles, giving something back to the community even in retirement. Good on ya, Jim! Don’t let the haters get you down.
That’s just PLoS ONE! And while it publishes stuff without regard to impact, even it can and does reject if there are mistakes. I’ve reviewed twice for them, one accept and one reject. The other PLoS journals are mostly quite selective.
PLOS ONE publishes anything methodologically sound, but PLOS Biology, PLOS Medicine, and so on are all very selective journals. I believe PLOS Medicine rejects more than 80% of submissions.
Even PLOS ONE is not comparable to arXiv, since it at least involves peer review; it has a rejection rate around 30%.
Interesting; thanks for the information.
This is going to be uncomfortable, but I’ll say it anyway. There are parts of the world where it is rare to find a female scientist. Female doctors in the Middle East, for example, will in many places only work in women’s hospitals (which are not like the Brigham). With more open access journals and the internet, the men can review in their spare time, and it is with surprise or even outrage that they find women (unsupervised!) writing about men. That’s not an excuse, it’s just the way the world is. They’re upset, we’re upset. But let’s not get all Hegelian about it.
Robert:
What do you mean, “Hegelian”?
Assuming an inevitable evolution (into being like us). I’m not saying anyone’s doing that but I suspect the reviewer’s identity will come out in the end and I’m wary of xenophobia. If journals are international then we have this problem of culture clash that is not going away any time soon.
Peer review is the art of running the monkey cage from the monkey cage.
See this paper:
http://www.nber.org/papers/w12369
I always thought in benefitted, in “laugh-them-out-of-the-room” terms, from the fact that the first author’s name appears to be female. (In fact, both authors are male.)
If there’s anything laughable in the paper, it’s not apparent from the abstract.
What do you think really happened here? p-value -> story time -> betray idealogical bias -> irritate male reviewer?
Arxiv does not claim to do any vetting. I think PLoS does. To some extent.
If a reviewer had suggested adding a woman’s perspective on an all male author list, would we have seen a similar reaction? I wonder.
You’re right that it’s not symmetric, which is the whole issue: it’s just incredibly tone deaf in this debate to imply that including men as authors will naturally make the article less ideological and ungrounded in empiricism.
Well, the symmetric situation would be a female reviewer telling two male scientists that their paper was of low quality (though providing few details) and that there’s no point in even trying to publish without getting a woman’s buy in.
You’d absolutely see a similar reaction–obviously some of the people complaining would be different (though by no means all) but still.
Thinking through all the differences and keeping it symmetrical is hard, partly because I think it’s more likely to be cleaned up “in house.” It’s hard to imagine an appeal getting turned down. Despite the outrage I think to some extent a male scientist making these comments is viewed as primarily “insensitive”–maybe there *were* methodology problems he saw, however much of a pig he was when responding?–while a female scientist saying this sort of crap is instantly incompetent and/or delusional.
Yes it would be ridiculous as well, it’s just instead of this discussion it would be gamergate style anti-feminist craziness.
Suggesting incorporating or responding to different perspectives is one thing, suggesting that the research team is incompetent due to the genders of its members is another.
As per the other recent thread, diverse research teams probably do have some special, positive traits, and it might well be worth the effort to do a systematic meta analysis of the relationship between team member gender and findings. That’s not anything like what happened in this instance however.
Who cares if it’s symmetric? or sexist? or if diversity helps? All that is besides the point. If Ingleby wants to write a paper and I want to read it then why does anyone else have a say in the matter?
Put it this way. If I want to watch a movie I watch it. If I don’t want to waste so much time on bad movies, I may first look at a movie review from a critic whose judgment and taste I trust before deciding to go.
Now imagine we replace that fine solution with the following system. I show up at the movie theater. An anonymous randomly chosen idiot from a pile of idiots is chosen to review the movie. If they don’t like it enough, I’m denied admission to the theater.
The later system is basically peer review.
Well, she’s free to post her paper online on her website (or something similar to ArXiv) and you to read it.
Nobody is compelling you to read peer reviewed Journals.
You’re being purposely obtuse. You know it’s not that simple when her career depends on peer reviewed publications and when such publications are widely assumed to have a ”correctness” that others lack.
If everyone submitted papers to a central repository (the Smithsonian or something) and then allowed critics to review them the way siskel and ebert reviewed movies then problem solved. Youtube is full of reviewers who were never picked for the job. Here’s one for example:
https://www.youtube.com/user/emergencyawesome
He has over 600,000 subscribers. No one hired him or would have hired him. He has zero qualifications and is a total armature with a lame day job. To my mind he’s the least likeliest person to ever be a star critic/reviewer. Yet based the usefulness of his reviews, him and dozens of other self made critics/reviewers have a large following and impact the success of movies and TV programs.
It’s extraordinarily unlikely the best peer reviewers will get that job. Even if they did how would we know? Everything is anonymous. For the reviewing service to be useful, we need to know who the reviewer was and their track record for finding gems.
I still wouldn’t give even the best reviewers any say over what makes it into the central repository though. I don’t think it’s a coincidence that the fetishizing peer review has increased at a time when science is stagnating.
Hear Hear!
Even PLoS ONE has a significant rejection rate, and some PLoS journals are highly selective.
PLoS Medicine’s impact factor is 14. They have an 80% rejection rate. In some fields, PLoS Medicine > PNAS.
People keep saying this as if it were a good thing.
The rejection rate? Or the impact factor? Or the “> PNAS”?
Off topic but not sure if you saw this:
http://www.nature.com/news/first-results-from-psychology-s-largest-reproducibility-test-1.17433
Nick,
Yes, I was involved in that project and I heard about this.
The post had this quote by Ingleby:
Which makes me wonder, why are reviewers provided the authors’ names and affiliations in the first place? Is there a good argument to let the reviewers know this info.?
Rahul:
To me, the main thing is that blinding a paper can be a pain in the ass. Also, looking toward the future: I think a natural future for scientific publication will be for everything to be posted online (on some sort of generalized Arxiv that isn’t so focused on physics and math), and then journals will become curators rather than publishers. The editorial board of the Journal of the American Statistical Association, for example, would read the new papers that are coming out online and then, every three months or whatever, they’d give a list of recommended papers, with links. No need for the journals to be publishing; they can just curate. And of course this would be coupled with post-publication review.
In that scenario, nothing’s blind unless the authors choose to publish their papers anonymously. So, in that sense, I think blind review is a bit of a dead end.
In the future, scientific papers will become blog posts and reviews will become comments or blog entries of somebody else. Journals will change to aggregation sites. Then scientific publication will reach the levels of journalism. Oh well…
It depends on the discipline, but in a lot of medical research the exact location and nature of the centres recruiting patients is relevant to how the findings are interpreted: different countries, regions and hospitals have different demographic patterns, healthcare systems, levels of surgical expertise etc… Once you know institutional information then in many cases anyone in that field would be able to guess who the authors are anyway. Many authors are also noticeably biased in citing their own work, although this can of course sometimes be justified. Factors such as these (among others) mean that full double-blinding is not as straight-forward as just clipping off the title page.
I read that PLos One have apologised and fired the academic editor, which sounded harsh, until I read this: In an email to the authors on 27 March, the journal rejected the paper on the grounds that “the qulaity [sic] of the manuscript is por [sic] issues on methodologies and presentation of resulst [sic]”. Via http://www.timeshighereducation.co.uk/news/sexist-peer-review-causes-storm-online/2020001.article Now I understand why they are so keen to shut down this story rather than just blame the reviewer.
High rejection rates are the hallmark of bad science.
To judge a journal by its rejection rate is like judging a school by its SAT scores, except it is worse.
I don’t really understand this. Given how annoying it is to submit, why would anyone submit to journals with high rejection rates, unless the rewards (the readership reach, the community perception of your work, etc.) for acceptance were high?
The rewards are high aren’t they? A Nature paper gets you far higher career rewards, chances of popular media coverage etc.
Haven’t you answered your own question?
To be fair, those who write to be published are already statistical outliers.
Re: ““Marginally better health and stamina”—that’s a laff and a half! … that men die at a higher rate than women at every age.”
Not necessarily a contradiction. Say men and women have the same “life energy” but men burn through it faster than do women, i.e. buying greater health and stamina in each given year, but shortening their lifespan. I’m not saying it’s true, I’m saying it’s possible.
“doesn’t realize that men die at a higher rate than women at every age.”
Not true, at least if “men and women” is taken to include boys and girls
Ages 10 and 11:
Male death probability Female Death probability
10 0.000082 99,155 66.74 0.000090 99,304 71.50
11 0.000086 99,147 65.75 0.000096 99,295 70.51
Source http://www.ssa.gov/OACT/STATS/table4c6.html
For the USA, obviously.
Siegfried:
Wow—that’s weird about ages 10 and 11, I’d never noticed that before! Really strange.
I don’t think this affects my point, because when the reviewer wrote that thing about “marginally better health and stamina” he was clearly talking about adults. But, still, that’s an interesting bit of data.
I’ve not seen a good explanation. Plausible guess: stress from puberty is greater in girls, especially early puberty.
The reviewer was right: It’s the stress from trying to write all those scientific papers. They should let the boys do the intellectual heavy lifting in middle school!