Why I Decline to do Peer Reviews (part two): Eternally Masked Reviews

In addition to the situation described in a previous post, there is another situation where I decline to do a peer review. First, I need to define a couple of terms. “Blind review” refers to the practice of concealing the identity of reviewers from authors. The reason seems pretty obvious. Scientific academia is a small world, egos are easily bruised, and vehicles for subtle or not-so-subtle vengeance (e.g., journal reviews and tenure letters) are readily at hand. If an editor wants an unvarnished critique, the reviewer’s identity needs to be protected. That’s why every journal (I know of) follows the practice of blind review.

“Masked review” is different. In this practice, the identity of the author(s) is concealed from reviewers. The well-intentioned reason is to protect authors from bias, such as bias against women, junior researchers, or researchers from non-famous institutions. Some journals use masked review for all articles; some offer the option to authors; some do not use it at all.

A few years ago, I did a review of an article submitted to Psychological Bulletin. The journal had a policy of masked review posted on its masthead, noting that that the identity of the author(s) is concealed from the reviewers “during the review process.” I liked the article and wrote a positive review. The other two reviewers didn’t like it, and the article was rejected. I was surprised, when I received my copy of the rejection letter, that the authors’ identity was still redacted.

So I contacted the editor. I was sure there had been some (minor) mistake. But the editor refused to reveal who the authors were, saying that the review was masked. I pointed out the phrase in the statement of journal policy that authors’ identity would be concealed “during the review process.” I had assumed this meant, only during the review process. The editor replied that while he could see my point, he could only reveal the authors’ name(s) with the authors’ permission. This seemed odd but I said ok, go ahead, ask the authors if I can know who they are. The answer came back that I could, if I revealed my own identity!

Now, I should not have had any problem with this, right? My own review was positive, so this was probably a chance to make a new friend. I only wanted to know the authors’ identity so that I could follow their work in general, and the fate of this particular article in particular. Still, the implications disturbed me. If the rule is that author identity is unmasked after the review process only if the reviewer agrees to be identified to the author, then it seems that only writers of positive reviews would learn authors’ identity, because they are the only ones would agree. Authors of negative reviews would be highly unlikely to allow their identity to be revealed because of possible adverse consequences – recall this is the very reason for “blind” review in the first place. And, the whole situation makes no sense anyway. What’s the point of continuing to mask author identity after the review is over?

At this time, ironically, I was a member of the Publications and Communications Board of the American Psychological Association, which oversees all of its journals including Psychological Bulletin. And then, though the normal rotation, I became Chair of this august body! There was a sort-of joke around the P&C Board, that every Chair got one “gimme,” a policy change that everybody would go along with to allow the Chair to feel like he or she had made a mark. The gimme I wanted was to change APA’s policy on masked review to match what the statement at Psychological Bulletin implied was its policy already: Authors’ identities would be revealed to reviewers at the conclusion of the review process.

The common sense of this small change, if that’s what it even was, seemed so obvious that arguments in its favor seemed superfluous. But I came up with a few anyway:
1. The purpose of masked review, in the words of the APA Editor’s Handbook, is “to achieve unbiased review of manuscripts.” This purpose is no longer served once review is over.
2. Reviewers are unpaid volunteers. One of the few rewards of reviewing is early and first-hand contact with the research literature, which allows one to follow the development of research programs by researchers or teams of researchers over time. This reward is to some extent – to a large extent? – removed by concealing author identity even when the review is over. Moreover, the persistent concealment of author identity signals a distrust of reviewers who have given of their time.
3. Important facts can come to light when author identity is revealed. A submitted article may be a virtual repeat of a previous article by the same authors (self-plagiarism), it may contradict earlier work by the same authors without attempting to resolve the contradiction, or it may have been written by a student or advisor of a reviewer who may or may not have noticed and may or may not have notified the editor if he or she did notice. These possibilities are all bad enough during the review process; they can permanently evade detection unless author identity is unmasked at some point.
4. The APA handbook already acknowledges that masking is incomplete at best. The action editor knows author identity, and the mask often slips in uncontrolled ways (e.g., the reviewer guessing – correctly or not). So ending masking at the end of the review process is a way to equalize the status of all authors rather than have their identity guessed correctly in some cases and incorrectly guessed in others — which itself could have odd consequences for the person who was thought to be the author, but wasn’t.

Do these arguments make sense to you? Then you and I are both in the minority. The arguments failed. The P&C Board actually did vote to change APA policy, as a personal favor I think, but the change was made contingent on comments from the Board of Editors (which comprises the editors of all the APA journals). I was not included in the Board of Editors meeting, but word came back that they did not like my proposal. Among the reasons: an author’s feelings might get hurt! And, it might hurt an author’s reputation if it ever became known that he or she had an article rejected. Because, it seems, this never happens to good scientists.

Today, the policy at Psychological Bulletin reads as follows: “The identities of authors will be withheld from reviewers and will be revealed after determining the final disposition of the manuscript only upon request and with the permission of the authors.” This is pretty much where the editor of the Bulletin came down, years ago, when I tried to find out an author’s identity. I guess I did have an impact on how this policy is now worded, if not its substance.

So here is the second reason that I (sometimes) decline to do peer reviews. If the authors’ identity is masked, I ask the editor whether the masking will be removed when the review process is over. If the answer is no, then I decline. The answer is usually no, so I get to decline a fair number of reviews.

Postscript: After writing the first draft of this blog, I was invited to review a (masked) article submitted to the Bulletin. I asked my standard question about unmasking at the conclusion of the review process. Instead of an answer, I received the following email: “As it turns out, your review will not be needed for me to make a decision, so unless you have already started, please do not complete your review.” So, I didn’t.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s