I get about 10 requests to referee journal articles each week. At this point, even the saying No part is getting tiring. I think I’d much prefer Kriegeskorte’s system of post-publication review where whatever you write about a paper is open and available to all to read, and where you can devote your review efforts to papers worth reviewing (either because of their inherent quality or importance, or because they’ve been hyped and need to be corrected).
But we are locked in a prisoner’s dilemma game. Who’s going to be the first to pull the trigger and stop publishing in peer review venues? If we all did it at once, we’d be good, but as it is…
Sometimes it seems that too much is being published, which I suspect is the uderlying problem. I would almost rather that tenure requirements and publication standards encouraged researchers (at least in the policy and public policy world I know) to spend more time “finishing” their product before publishing. I think some changes could enhance the quality of research, quality of what is published, and the social utility. For instance, it is very hard to get data sets for most published articles (in my experience), there should be a standard set for how to prepare a data set for the public. Time consuming, and perhaps of little marginal utility to the authors, but seriously needed and so it should be rewarded or required. Another example: far too many publicatins in social sciences involve a year of data when clearly a few more years would make for a stronger paper. Again, it takes time, but what value is there in a paper that is just in the hackneyed styles of “here’s a regression that’s mildly different.” The third suggestion, off the top of my head, is to get scholars to demonstrate that they have already had diverse set of readers give comments. Again, with fewer papers floating around (albeit perhaps longer or more complex) maybe things would lighten up a bit, and improve the quality at the same time.