How Not to Be Reviewer #2

For those of you unfamiliar with the academic meme, Reviewer #2 is the embodiment of all that is wrong with the peer review system. No, wait, all that is wrong with the peer in peer review.

At the time of writing, the Reviewer 2 Must Be Stopped Facebook page has nearly 7,000 members which goes to show reviewer 2’s assholery is recognised by at least 7,000 people. Given the relatively niche nature of academic publishing, 7k is a lot. Now, to be clear, the reviewer 2 I refer to is not the same as Tenure She Wrote’s reviewer #2. As Tenure She Wrote writes (heh, Wrote writes looks odd):

Now that I have graduate students of my own, I’ve been thinking about how to train them to be reviewers, without creating either mice* or attack dogs**. If Reviewer #1 is that ineffectual, unconfident reviewer, and Reviewer #3 is too angry and aggressive, I’d like to be just right***: Reviewer #2.

For Tenure She Wrote, Reviewer #2 is the happy medium between the too hot and the too cold porridge bowls of academia. Reviewer 2 seeks to be helpful, firm, and not aggressive. As she goes on, Tenure says that we don’t actually train graduate students, postgrads, or early careers on how to review, so can we really be surprised that so many bad reviewers exist? Tenure has a great point, one which I will come back to later, but for now, let us focus on Reviewer 2.

Who is Reviewer #2? Literally, Reviewer 2 is the anonymised moniker given to the second peer to review a research paper. In common parlance, Reviewer 2 can be summarised as possessing the following qualities:

• Grumpy
• Aggressive
• Vague
• Unhelpful
• Overbearingly committed to a pet discipline
• Overly focused on a particular methodology
• Inflexible
• Unwilling to give the benefit of the doubt
• Unwilling to view the authors of a submitted paper as peers.

If you need examples, go on over to Shit My Reviewers Say. Just a quick scroll through is enough to make non-academics shake their heads in sympathy.

So, to get back to Tenure She Wrote’s point that reviewers are not taught how to be reviewers, and most publishers include only the vaguest suggestions on how to review a work, I’m going to use the rest of this post to share my advice on how to be a good peer, and a good reviewer.

Step 1: Make sure you are the right person for the article.

Most reviews I have been asked to do, I have been asked. I have yet to be forced to review an article. Guilt tripped, maybe, but not forced. If I don’t feel confident in my knowledge of a particular methodology, or that I don’t have expertise with a particular field of literature, I’ll politely decline to review.

Now obviously you don’t have to be an expert in every aspect of a paper in order to review it, but you should have some knowledge of its themes and methods. If you have never used the method but studied it at a Masters level, you’re probably competent enough to review the article. Likewise, if you remember reading a few of the authors in the bibliography, you can probably appropriately assess how the article fits in with the current body of literature and whether or not it has contributed anything original to the field. Obviously this is a rough guide and you need to be your own judge of your abilities.

Step 2: Make sure you are in the right frame of mind to review.

Since, as we established above, you probably aren’t being forced to review a paper, don’t say yes to every paper that comes your way. Likewise, don’t say yes if you know you’re going to be snowed under with marking, or if you have study boards, or if you’re examining a PhD, or if you have a grant bid due in, or if you are doing any number of tasks academics do during the year which cause us to be immensely grumpy… If you are stressed and pressured, that will absolutely come through in your review and what’s worse, it will effect the quality of the review.

Reviewing the work of your peers should be pleasurable. Don’t laugh. I am serious. It should be a chance to see what others in your field are doing, a chance to read cutting edge research, and a chance to share your expertise (what good is knowledge if you don’t use it?) When I review, I do it in stages. In the first phase, I read through the paper and take small notes on the side. I do this in the bath, in a park, or at a restaurant with a glass of something. It is a nice, peaceful excuse to enjoy some time alone and remember why I became an academic. The second round of review, I re-read my comments and formulate them into what I think will be most useful for the author(s). The third phase, I type up my responses and include links where applicable. It is time consuming, but not as bad as you think.

Step 3: Review as you would want to be reviewed.

I approach papers as I would approach marking an external PhD. I, largely speaking, give the authors the benefit of the doubt. They didn’t cite author X? Well, maybe they aren’t aware or maybe they didn’t feel X fit in with what they are trying to achieve. Rather than write something along the lines of, “The authors clearly aren’t aware of the existing work of X and are thus re-inventing the wheel” I write, “Citing X in paragraph 2, page 4 would probably support this aspect of your argument” or “I’d like to see engagement with theorist X”.

I can’t think of a single example when taking a heavy handed, aggressive, or admonishing tone would be appropriate in a peer-review. It makes authors feel bad and it makes you look like an asshole. No one wins because, like lobsters in a bucket, tearing someone down doesn’t put your career any higher.

Don’t get me wrong. We have all had a frustratingly bad paper which suffered so many problems its impossible to start listing them. Now having said that, if you feel genuinely frustrated by how bad a paper is, take your officemates out for coffee and have a rant. Don’t put your malice in the review because best case scenario the authors will feel frustratingly misunderstood and offended, worst case scenario, they will feel horribly depressed, rejected, and might even give up. Either way, it really won’t be a constructive outcome.

Step 4: Be specific, be helpful.

A recent conversation with a colleague called to my attention the difference between high and low comments. High comments focus on things like theme, appropriate use of literature, methodological goodness-of-fit. Low comments concern things like grammar, spelling, maybe even structure. Most reviewer comments should focus on high comments with the understanding formatting and spelling will likely be handled by a copy editor.

Even still, high comments should be specific. If you take issue with the analysis, don’t write “analysis problematic”. Say why, say what it is missing, suggest alternatives. Likewise, don’t say “doesn’t engage with body of literature”, say which it doesn’t engage with.

If you suggest authors include a piece of work, make the effort to include a full citation. For example, if you were going to recommend they include my new book, Brown 2015 wouldn’t be specific enough to be useful. Even if the authors managed to work out you were suggesting my work, I have 4 publications in 2015. Which did you mean? When possible, give a link.

So in conclusion, be sure you are the right person to do the review, that you are in the right frame of mind, and approach the review with the goal of being helpful. If we all did this, the peer review process sure would be a lot nicer. To paraphrase a quote from the great Bob Ross, “I’d like to wish you happy peer-reviewing, and God bless my friend.”

Until next time, review in peace,

Ashley