Increasing Transparency in Manuscript Review (Part 1)
It wasn’t long ago that I wrote in favor of the anonymous review process held as the gold standard for peer review of academic work. In order to reduce the amount of error in the judgments rendered, I suggested that anonymizing reviews worked to make reviewer judgments better. But since September 2011, I have had the pleasure of working as the editorial assistant for an academic journal edited by one of my professors, and this experience has given me a new perspective on this particular process. So while my logic in favor of anonymous review still stands, there is a lot of room for increased transparency in the manuscript review process.
Today, the review process usually works something like this. An author creates an account on a journal’s website and goes through some standard forms in order to submit a manuscript. Once the paper is submitted, the waiting game begins. After some period of time (often quite a long wait), the author receives an email from an editor rendering the journal’s judgment. The author has little recourse after the judgment is rendered, as appeals are rare and the process seldom well-defined. From that decision, thus, the author must move forward; only in the case of an acceptance are the next steps perfectly clear. In short, it’s a confusing process shrouded (somewhat intentionally) in mystery.
This mystery, however, does nothing to increase confidence in the review process. First, because the process is so opaque, communication with journal editors tends to be overly formal, even to the point of reducing the effectiveness of the communication. Authors feel the need to treat editors (and even their lowly assistants) with excessive respect. This suggests that authors feel intimidated by the journal and its omnipotent editor. Second, authors are accustomed to receiving no information about how the review process is proceeding until they receive final judgment from the editor. Only in cases of excessive delays will authors write to inquire about the status of their manuscript. And I suspect, from personal experience, that this is done with a sense of trepidation – “what if I jinx it?!”. If reviews are seen as such fickle things (dictated by the bipolar whims of the editor), then keeping the process secret does nothing to change this reputation.
It is reasonable for authors to have a distrust of the review process because everything is quite murky. And as authors serve as reviewers for manuscripts, no greater clarity is introduced into the process. Reviewers do not regularly receive feedback on their reviews, either from an editor or by being able to see the opinions of other reviewers. Thus a reviewer recommending “acceptance” does not know if other reviewers felt the same way, at least not until the manuscript is actually published. Because the process of submission to review to publication can take up to several years, reviewers may not even recognize the piece when it comes out. Why all the delay? Reviewers aren’t given any more reason for it than authors are.
Working as an editorial assistant gives one a whole new perspective on the process, so much perspective that it is too bad many professors see editorships as highly prestigious yet highly undesirable (too much work!). From an insider’s perspective, it is easy to understand the delays. It takes sometimes several weeks for the editor to deliver a preliminary judgment on a manuscript (should it go out for review?). Next, reviewers can be awfully slow sometimes, missing deadlines by months. But the editor wants to give reviewers leeway (after all, they have more power than the editor in some ways), so reminders to those reviewers may only go out after the initial deadline is missed by three months or more. And reminders often don’t work; initial reviewers will say they are now too busy, or make promises that they don’t keep. Once reviews are in, the editor needs a large amount of time to carefully review the judgments rendered, review the manuscript itself, and then make a final decision to be relayed to the author. If the journal receives a large volume of submissions, this entire process can take massive amounts of time to complete.
Given these constraints, why aren’t editors more open about the entire review process? After all, it seems that being more open about how hard the work is would make for a smoother time for editors and authors. I’ll consider the advantages of an opaque review process tomorrow.
Related posts (automatically generated):