The Value of Anonymous Review
I’m just not into being anonymous. Though my first email address (at Hotmail.com) used a handle, it wasn’t long after that I started to prefer addresses that used my name. You can see that preference in the exceedingly boring name of this blog. So when it comes time to give feedback to others, I like to sign my name to what I write. In the classroom, students are clear that it is me giving the feedback; even when working as a grader this semester, I went to the classroom and introduced myself to the students. When writing up course evaluations, I sign my name as well. Overall, I find that the lack of anonymity changes the way I give feedback, as I become more aware of how the feedback reflects on me.
But the standard for reviewing for papers or journals in the academic world is one of anonymity. A manuscript is sent out to reviewers sans author names, and the review comes back without the reviewer’s name. Whether required by the journal or simply optional, anonymity seems to be the standard practice. Though I personally have never been assigned to review a journal article myself, I did assist a professor with a review last month, and it set me to thinking: when it comes time for me to review, what choice should I make? In cases where anonymity is optional, should I sign my name?
I put this question to my professor, and this is what she said: “As for signing reviews, I strongly urge you NOT TO DO IT. It is critical that the review process is double-blind, because we humans are not objective creatures and the academia is a small world. If you sign your reviews, you’ll create friends and enemies, and that will impair the objectivity of these people when they are reviewing your work.”
Some excellent points there, and it got me to thinking: what is the purpose of a review, and how does the addition of a signature and the removal of anonymity help or hurt the process? Thinking about reviews from a testing perspective, we can see a review as an attempt to capture whether or not a manuscript should be accepted for publication. Thus the formula for a review becomes this:
Should Be Published = Reviewer’s Judgment + error
Ideally, the reviewer’s judgment is correct all the time, but there’s good reason why manuscripts are usually sent to two reviewers. The error in the process keeps us from full confidence in the judgment of just one person.
This formula works whether anonymous or not, especially because the editor (who knows the author of the manuscript and the reviewers) makes the final decision. But anonymity comes into play when considering the more likely outcome of a journal submission: either rejection or the very common “revise and resubmit.” In both these cases, the comments from the reviewer are very important because they give guidance to the authors on how to revise the paper so that it can be published, either in the same journal (as in the case of “revise and resubmit”) or another journal (in the case of a rejection). In these cases, the feedback is more important than the final decision, requiring us to revise our equation.
Correct Suggestion = Reviewer’s Suggestion + error
The author must make a decision about which comments to accept and act on and which to ignore. This is a tough decision, as the author knows that the reviewer’s suggestions may or may not be helpful; error is ever-recognized in feedback of this type. Removing anonymity from the process introduces more error because it forces the author to consider both the merits of the feedback per se and also the weight to give the feedback overall based on the assigned reviewer. If the reviewer is a renowned scholar in the field, then the comments may be more readily accepted. If the reviewer is unknown, or worse, some kind of clown, then the author may have a more difficult time deciding what to change in the manuscript.
In other words, removing anonymity introduces error into the review process. It adds in factors that are unaccounted for in the equation above. If anonymity is removed, the reviewer’s suggestions must be properly weighted to account for the credence given the reviewer by the author. As a top scholar can give a bad review and an unknown or disrespected scholar a good one, this weighting by reputation is likely to be fraught with problems. It undermines the effectiveness of the system (one in which authors must begin to make their own best decisions about what to do with their manuscripts, such that over time, their scholarship – and all scholarship – improves in quality), and alters the perception of reliability in the reviewer’s suggestions.
Based on these reasons, I conclude that I will respect the anonymous nature of the review system, even if I’m still upfront about who I am in all other interactions. From a testing theory perspective, removing anonymity does more harm than good.
Related posts (automatically generated):