At A Crossroads: Peer Review
by Chris Jordan
Recently, it seems that there is a growing number of complaints coming out of academia regarding getting work published at good venues and the importance of having a high publication count. I am sure many of us have heard the phrase "publish or perish" before. In fact, there has been a case of a professor leaving his field of research (http://www.cs.utah.edu/~michael/leaving.html) over these issues. He criticizes the peer review process as being both fallible and corrupt in his explanation of why he left. He also discusses what he believes to be misplaced importance that many people have put on publishing at top conferences.
Obviously, publishing at top conferences such as an ACM Special Interest Group (SIG) is very challenging. There is a lot of competition to publish at those venues as papers are submitted to them from researchers all over the world. The sheer volume of submissions places a lot of pressure on the peer review process. To insinuate that some reviewers have corrupted this process because they are trying to publish their own work is a little offensive though. Reviewing and selecting papers for a large conference is a monstrous task. Reviewers and committee members are the ones that bear this burden by volunteering large amounts of their own time; they deserve a little more respect than having their academic integrity questioned.
Admittedly I can understand to an extent why there are these complaints about the peer review process. We are placing a fair bit of trust in reviewers to thoroughly and objectively review all submissions. Of course reviewers are subject to making mistakes or misunderstanding the work presented in a paper they are evaluating. As well, sometimes reviewers are given papers that are not really in their area of expertise. The peer review process is not perfect but that is why there are multiple reviewers. As well, if an author feels like their paper was not properly reviewed, they can contact the conference committee who can ultimately overrule the decisions of the reviewers.
Reviewers that take advantage of their position and corrupt the peer review process are taking a huge risk with their own careers. If someone were found undermining the reviewing process, it would essentially ruin him/her. That being said, I cannot imagine how to go about proving that a reviewer acted maliciously unless he/she was being completely obvious. So it is possible to get away with rejecting papers for personal gain. When considering though, the jeopardy a person would place his/her career in from committing such an act and the actual tangible gain, it is hard to imagine many people willing to engage in this practice.
Everyone involved in a conference has a strong interest in the peer review process preserving its integrity. When members of the community believe that a conference is losing its integrity then it loses its value. It is understandable though why the peer review process comes under fire as these days there is a lot of pressure on professors and grad students to publish. A person's publication count and where he/she publishes can play an important factor in getting positions at universities and winning grants. These conditions make it difficult to determine if it is the peer review system that is actually breaking down or if it is that people are concerned about their careers.
Unless you have been clearly wronged, it is probably best to not accuse the peer review system of being completely flawed. Of course there are problems with peer review but any system that involves people is subject to human error. It is my opinion that having more reviewers per paper is a good thing because that means more chances for constructive feedback to improve your work. Peer review is not perfect but it works. Try to take rejections in stride and remember that everyone gets rejected; it is part of being an academic. What is important is that you keep on trying; that is being an academic.
Biography
Chris Jordan (chris.jordan@acm.org) is a PhD candidate in the Faculty of Computer Science at Dalhousie University. His research interests are in Information Retrieval, Natural Language Processing, and Web Mining. Website: http://www.chrisjordan.ca