Peer review of academic research is at the heart of publishing. It is important that this process is not tainted by reviewer bias. Two popular modes of review exist. In single-blind peer review, the authors do not know who the reviewers are. The reviewers know who the authors are. In double-blind peer review, neither authors nor reviewers know each other’s names. Single-blind peer review is the traditional model. However, both models exist to eliminate bias in peer review.

The Physics Experiment

At the start of 2017, the Institute of Physics (IOP) gave authors the option to choose double-blind peer review. This option was available for Materials Research Express and Biomedical Physics & Engineering Express. Over the first seven months, 20% of authors chose the double-blind peer review option. Authors from India, Africa, and the Middle East were most likely to request the option.

IOP data indicates that more papers received rejections under the double-blind model. About 70% of papers received a rejection in the double-blind peer review process. On the other hand, only 50% of papers received rejection under single-blind peer review. The difference could be due to reviewers assuming that authors requesting this option had written poor papers. It could also be due to reviewers acting more objectively. However, authors in the double-blind trial were satisfied and felt it was the fairest approach.

Bias in peer review is a real problem. There have been many studies showing that women and minorities are less likely to get published, funded, or promoted. This bias can be both conscious and unconscious. Within scientific publishing, this means that fewer women are asked to review papers. It also means papers by women are cited less. There are two peer review models where identities are hidden. Which is more likely to get rid of bias?

Double-Blind vs. Single-Blind Peer Review

The 2017 Web Search and Data Mining conference provided a good opportunity to experiment this theory. In Computer Science, papers often appear first (or exclusively) in peer-reviewed conferences. The program committee decided to randomly split its reviewers into two groups. One would serve as double-blind peer reviewers. The other as single-blind peer reviewers. The experiment would help decide which approach might have more bias.

The authors found that there were differences between the review groups. All reviewers had access to paper titles and abstracts. Based on this, reviewers indicated which papers they wanted to review. The single-blind reviewers requested to review 22% fewer papers. Single-blind reviewers were also more likely to choose papers from top universities or IT companies to review. They were also more likely to give a positive review to papers with a famous author.

Single-blind reviewers have access to the authors’ names and institutions. The study indicates that author institution had a significant influence on single-blind reviewers’ decisions to bid for a paper. There was no detected bias against female authors for this conference. A metareview combining this conference’s data with other studies indicated that there was a significant bias against female authors.

The Web Search and Data Mining conference experiment show that single-blind reviewers use information about authors and institutions in their reviews. It could be that this information is helping the reviewers make better judgments. It could also be that this is putting work from non-prestigious institutions and authors at a disadvantage. Two papers of equal value may be rated differently by single-blind reviewers based on who wrote the paper.

A Review of Peer Review

On the other hand, double-blind peer review provides a false sense of security. Well-known authors can be easily identified by the nature of their work. The paper may also make reference to previous work that they published. There may be other clues as well, such as a preference for a technique or compound. This means that, even without the names, reviewers can figure out who wrote a paper. It would, therefore, be better to tell the reviewer who wrote the paper and ask if there is a conflict of interest.

The actual process of removing author information to hide identity fails 46-73% of the time. The problem isn’t identifying the author. The problem is whether reviewers have a prejudice against authors from a certain country, race, or gender? While the focus has mainly been on reviewers, very little discussion exists about biases of editors. Editors, after all, have the final say.

Peer review is part of the academic research cycle and it is clear that there is bias in this process. Reviewer bias often affects women, minorities, and researchers from non-prestigious institutions. In order to try and fight this problem, journals use blind peer review. However, single-blind peer review gives the advantage to well-known authors. Double-blind peer review may not actually eliminate bias, hence researchers feel that it is better to switch to open peer review.