Rob Bloomfield writes:
I [Bloomfield] am a new editor of a new Journal of Financial Reporting, and we’re trying to make some changes to peer review in our field. I’d be very interested to hear your and your readers’ thoughts on whether our approach will help address the types of problems often discussed on your blog.
I’ve laid out the approach in two short videos and a more detailed pitch to authors (https://ssrn.com/abstract=2913262).
Below are the video links with short snippets of transcript. The full transcript is included on youtube for those who don’t care for video.
Substance, Commentary and Compromise: JFR’s New Approach to Academic Publishing (https://youtu.be/Qf8FkuBMfus)
Starting a brand new journal gives us a chance to make some real changes in how papers are written and published.
Tell me if you’ve heard these common concerns:
- Journals ask for novelty, but they don’t treat it kindly. They distort new ideas to fit old thinking, or simply reject them.
- Journals ask for rigor, but don’t re-examine prior claims. They leave the first word the last word.
- Journals promise a level playing field, but authors without connections and prestige find it hard to publish good work.
These problems aren’t secrets, but journals have little incentive to change once they are well established. That’s where JFR comes in, and here’s our fix.
First, we recognize that every paper must compromise on some goals to achieve others. It’s hard to explore new ideas and test old ones. It’s hard to identify causality and generalize across samples and settings. It’s hard to address deep issues and provide useful applications. We’re fine with compromises; in fact, we encourage them. We’ll let you pursue a limited set of goals, as long as you do it well and report honestly about what you did and didn’t achieve.
Second, we’ll base publication decisions on the substance of your work—what you actually do and find. How do you gather data? How do you analyze it? What narrow conclusions do you draw? We’ll hold substance to the highest standards. But most papers also include a good deal of commentary. What makes the study interesting? How does it address big questions? How does it test or extend existing theory? How might it affect practice or policy? These issues are important, but they’re so hard to evaluate that a lot of good papers get rejected on the basis of reasonable disagreements about highly subjective matters. We think that’s wrong, and that this is one of the big reasons peer review is so tough on new work and unknown scholars. We won’t use reasonable disagreements as a basis for rejection. Instead, if need be, we’ll hash them out in published discussions.
By allowing compromise and reasonable disagreement, we at JFR can make a much stronger push for innovation.
The Journal of Financial Reporting Playbook (https://youtu.be/E7mWo0P9qZ8)
We have a different view of the publishing process. For any journal, you can think of a manuscript as a football; we score points by getting the ball over the goal line. All too often, journals make authors feel like their battle is against reviewers and editors. But nature is the real opponent: complexity, randomness and flawed data blur our vision and make it hard to measure and identify causes and effects.
Authors are the so-called ‘skill’ players who run, pass and kick the ball forward, by bringing new theory, data and analysis to the field. Reviewers aren’t working against the authors, they are helping them find a path through nature’s defenses.
Where are the editors? We’re the coaches over on the sideline, making sure that the authors and reviewers share a vision of how to move the ball forward. We’re also the umpires, making sure everyone is following the rules. Conflict of interest? Maybe. But welcome to peer-review.
JFR has a three-part coaching strategy.
- We’ll expand the playbook. Gather data however you can; draw your theories from any discipline; define financial reporting broadly; draw conclusions from Bayesian analysis, machine learning, or qualitative interpretation. Exploratory analyses, replications, null results—they’re all in our playbook.
- We’ll settle for field goals. Everyone’s looking for a touchdown. But it’s hard to explore new data sets and test theory; it’s hard to identify causality and generalize. It’s hard to address deep issues and be useful right away. We’ll let you pursue a limited set of goals, as long as you do it well, and are honest about what you did and didn’t achieve.
- We’ll go for two. OK, to make this metaphor to work, after a field goal you can get a couple more points by publishing discussions that run the ball over the goal line again, from right close up. And what a difference it makes! It’s not so much that we add commentary, but that we get to remove it from the paper itself.
Our goal is to accept or reject papers based on a narrow view of their substance: what did you do, and what did you find? We’ll hold substance to the highest standards. But everything else is commentary, and reasonable disagreements shouldn’t be a bar to publication. Instead, we’ll hash them out in discussions.
This gets at a lot of important issues in research communication. I’m usually better at using communication channels than at designing methods for improving communication more generally, so I appreciate learning about work in this area.
Also, I like the football analogies.