Why I Decline to do Peer Reviews (part one): Re-reviews

Like pretty much everyone fortunate enough to occupy a faculty position in psychology at a research university, I am frequently asked to review articles submitted for publication to scientific journals. Editors rely heavily on these reviews in making their accept/reject decisions. I know: I’ve been an editor myself, and I experienced first-hand the frustrations in trying to persuade qualified reviewers to help me assess the articles that flowed over my desk in seemingly ever-increasing numbers. So don’t get me wrong: I often do agree to do reviews – around 25 times a year, which is probably neither much above nor below the average for psychologists at my career stage. But sometimes I simply refuse, and let me explain one reason why.

The routine process of peer review is that the editor reads a submitted article, selects 2 or 3 individuals thought to have reasonable expertise in the topic, and asks them for reviews. After some delays due to reviewers’ competing obligations, trips out of town, personal emergencies or – the editor’s true bane – lengthy failures to respond at all, the requisite number of reviews eventually arrive. In a very few cases, the editor reads the reviews, reads the article, and accepts it for publication. In rather more cases, the editor rejects the article. The authors of the remaining articles get a letter inviting them to “revise and resubmit.” In such cases, in theory at least, the reviewers and/or the editor see a promising contribution. Perhaps a different, more informative statistic could be calculated, an omitted relevant article cited, or a theoretical derivation explained more clearly. But the preliminary decision clearly is – or should be – that the research is worth publishing; it could just be reported a bit better.

What happens then? What should happen, in my opinion, is that the author(s) complete their revision, the editor reads it, perhaps refreshing his or her memory by rereading the reviewers’ comments, and then makes a final accept/reject decision. After all, the editor is – presumably and hopefully – at least somewhat cognizant of the topic area and perhaps truly expert. The reviewers were selected for their specific expertise and have already commented on the article, sometimes in great detail. Armed with that, the editor should not find it too difficult to make a final decision.

Too often, this is not what happens. Instead, the editor sends the revised paper out for further review! Usually, this entails sending it to the same individuals who reviewed the article once already. Sometimes – in a surprisingly common practice that every author deeply loathes – the editor also sends the revised article to new reviewers. Everyone then weighs in, looking to make sure their favorite comments were addressed, and making further comments for further revision. The editor reads the reviews and perhaps now makes a final decision, but sometimes not. Yes, the author may be asked to revise and resubmit yet again – and while going back to the old reviewers a third time (plus yet new reviewers) is less likely, it is far from unheard of.

What is the result of this process? What an editor who acts this way would no doubt say is, the article is getting better and the journal is as a result publishing better science. Perhaps. But there are a few other results:

1. The editor has effectively dodged much of the responsibility for his/her editorial decision. Some editors add up the reviewers’ verdicts as if they were votes; some insist on unanimous positive verdicts from long lists of reviewers; in every case the editor can and often does point to the reviewers – rather than to him or herself – as the source of negative comments or outcomes.
2. The review process has been extended to be epically long. It is, sadly, not especially unusual for this process of reiteration to take a year or more. Any review process shorter than several months is considered lightning-fast for most journals in psychology.
3. The reviewers have been given the opportunity to micro-manage the paper. They can and often do demand that new references be inserted (sometimes articles written by the reviewers themselves), theoretical claims be toned down (generally ones the reviewers disagree with), and different statistics be calculated. Reviewers may even insist that whole sections be inserted, removed, or completely rewritten.
4. (As a result of point 3): The author is driven to, in a phrase we have all heard, “make the reviewers happy.” In an attempt to be published, the author will (a) insert references he/she does not actually think are germane, (b) make theoretical statements different from what he/she actually believes to be correct and (c) take out sections he or she thought was important, add sections he or she thinks are actually irrelevant, and rephrase discussions using another person’s words. The author’s name still goes on the paper, but the reviewers have become, in effect, co-authors. In a final bit of humiliating obsequience, the “anonymous reviewers” may be thanked in a footnote. This expressed gratitude is not always 100% sincere.

These consequences are all bad, but the worst is number 4. A central obligation of every scientist – of every scholar in every field, actually – is to say what one really thinks. (If tenure has a justification, this is it.) And yet the quest to “make the reviewers happy” leads too many authors to say things they don’t completely believe. At best, they are phrasing things differently than they would prefer, or citing a few articles that they don’t really regard as relevant. At worst, they are distorting their article into an incoherent mish-mash co-written by a committee of anonymous reviewers — none of whom came up with the original idea for the research, conducted the study, or is held accountable for whether the article finally published is right or wrong.
So that’s why, on the little box at the bottom of the peer review sheet that asks, “Would you be willing to review a revision of this article?” I check “no.” Please, editors: Evaluate the article that was submitted. If it needs a few minor tweaks, give the author a chance to make them. If it needs more than that, reject it. But don’t drag out the review process to the end of time, and don’t let a panel of reviewers – no matter how brilliant – co-author the article. They should write their own.