Refereeing, Formal Reviewing, and Reviewing

Summarizing the Differences Between Refereeing, Formal Reviewing, and Reviewing:


Refereeing follows a defined process and is based on criteria of originality, correctness, novelty, importance, and clarity of exposition. "Defined process" for refereeing means: The editor send an article to a fixed number (usually 3) independent referees.

Each referee prepares a detailed evaluation of the manuscript, which is then communicated (blind) to the author. The author almost always rewrites; the article is sent back to the original reviewers for final acceptance or rejection. This process usually takes 6 months to a year, often longer. Because of the depth of evaluation, followed by the refinement of rewrite, it is rare that this process can take place in less than 6 months.

Formal Reviewing

Formal reviewing also follows a defined process, similar to that of refereeing, but the criteria of originality, novelty, and importance are not required. The length of time for processing is usually shorter (2 to 6 months) since both evaluation and rewrite (if one is done) are based on fewer criteria.


Reviewing does not necessarily follow a defined process or set criteria. Its primary objective is independent assessment; formal reports do not have to be written by the reviewers, i.e., the review can be oral.

Refereeing is applied to all articles published in the primary journals (CACM, Journal, and the Transactions not Computing Surveys because the material, in general, is not originalÙ). Formally reviewing and reviewing are the evaluation processes usually applied to conference proceedings.

To know exactly how to categorize our proceedings, we would need to study the procedures and criteria for each, an onerous task not necessarily needed. For the time being, I think it is safe to say that although some of the proceedings may be refereed as defined above, most are formally reviewed or just reviewed. Speaking about ACM proceedings as "refereed" is probably misleading.

SIG Editor Manual
Last Update November 1993

The DevOps Phenomenon

ACM Queue’s “Research for Practice” serves up expert-curated guides to the best of computing research, and relates these breakthroughs to the challenges that software engineers face every day. This installment, “The DevOps Phenomenon” by Anna Wiedemann, Nicole Forsgren, Manuel Wiesche, Heiko Gewald and Helmut Krcmar, gives an overview of stories from across the industry about software organizations overcoming early hurdles of adopting DevOps practices, and coming out on the other side with tighter integration between software and operations teams, faster delivery times for new software features, and achieving higher levels of stability.

ACM Case Studies

Written by leading domain experts for software engineers, ACM Case Studies provide an in-depth look at how software teams overcome specific challenges by implementing new technologies, adopting new practices, or a combination of both. Often through first-hand accounts, these pieces explore what the challenges were, the tools and techniques that were used to combat them, and the solution that was achieved.

Why I Belong to ACM

Hear from Bryan Cantrill, vice president of engineering at Joyent, Ben Fried chief information officer at Google, and Theo Schlossnagle, OmniTI founder on why they are members of ACM.