# Reviewing Standards for CGO 2004

Reviewers often do not completely agree on a submission, so the detailed evaluation of the merits and deficiencies of a paper need to be summarized in the review. If you like the paper immensely, do not assume other reviewers will feel the same way. Describe in detail what you think is important about it and how it will contribute to theory or practice. If you are sure the paper should be rejected, you should explain why, politely , but in detail, because other reviewers may recommend acceptance. Even if the paper is not accepted, the work may be published elsewhere, and the authors will still benefit from your constructive suggestions.

The tone of your review is important. When you write an anonymous review, you are acting as a representative of our field. It is always possible to be constructive.

Papers submitted to CGO are expected to be original, innovative works. Papers should be no longer than 22 pages using 1.5 line spacing. Please see the following for the specific instructions given to authors on how to submit their paper blind.

The review process consists of four steps.

1. You should read the paper and enter your review information. You can continue to return and edit your review information until the end of the review period, as long as you have not "finalized" it.

2. When you are done with your review "finalize" it using the first menu selection, labeled "Are you done with your review".

3. When all reviews are finished, authors will be able to read the two sections labeled "Comments to be addressed in the rebuttal and Additional detailed comments to the author". They will then be able to respond to those comments. That is the only information that authors will see.

4. During program committee meeting, PC members will use all the review information and the information provided by the author to select papers for the conference.

When you review a paper, there are several things you need to tell us. This list follows the list of questions on the review form. When you change an entry on the form, you need to click the button labeled "Submit your paper review". If you leave the page without clicking that button, your review entry will not be stored.

- Are you finished with this review?
  Once you "finalize" this review, you can not modify it any further. Be careful with this option.

- Provide a short summary of the paper
  You should briefly summarize the paper.

- Strength of Paper
  Summarize in 1 to 3 sentences the strengths of the paper. You should be able to find at least some strength (positive point) in every paper.

- Weakness of Paper
  Summarize in 1 to 3 sentences the main weakness of the paper.

- Your qualifications to review this paper
  We realize that not all reviewers are equally qualified; accurately assessing your background will help the program committee understand your review.

- Writing
  Is the paper readable? If there are "show stopper" issues, you should inform the program committee.

- Relevance to CGO
  Do you think the paper would engender a lot of discussion? Do you think the average attendee would attend the paper presentation?

- Experimental Methodology
  Was the experimental method reasonable? Good benchmarks? Good assumptions?

- Novelty of paper
  This conference focuses on novel work; it's acceptable that a novel idea have slightly less stringent evaluation than an incremental improvement, so we need to know how novel the work is. If you think the paper is not novel, you should clearly state why and indicate related prior work in your comments to the author or program committee.

- Overall paper merit
  This is the single most important decision concerning the paper - do you think it should be accepted to the conference? If you think the paper should be a short paper, you must indicate that it would be an acceptable short paper and you should leave a comment to the program committee to that effect.

  1) Reject -
    Paper is not of interest to CGO, it has already been done, or has serious methodology flaws.
  2) Weak Reject -
    Paper is potentially of interest to CGO, and has some value in the ideas presented, but it has either potential methodology issues or it is a borderline incremental/already been done contribution.
  3) Weak Accept -
    Paper is of interest to CGO, and is an incremental contribution paper with average/weak methodology.
  4) Accept -
    This is of interest to CGO, a novel or new contribution with average/weak methodology, or incremental contribution paper that has good methodology.
  5) Strong Accept -
    This is of interest to CGO, and a novel or new contribution to existing area with good/average methodology, or an incremental contribution paper that has excellent methodology.
  6) Definite accept -
    This is of interest to CGO, and a novel or new contribution to existing area with excellent/good methodology.

- Comments for the authors to address in the rebuttal
  These are the comments the authors will concentrate on responding to in their rebuttal. If there are any important questions that need to be answered or you believe there are major issues with the paper, these all need to be raised here for the authors to address.

- Additional detailed comments to the author
  This is the single most important response you can provide to the author. One of the critical roles of a review process is to help authors understand any problems with their work so they can improve it. Please spend the time to indicate to the author and program committee why you've rated the paper the way you did.

- Additional comments to the program committee
  There may be reasons that influence your rating of the paper that would reveal your identity, or that you would only like made available to the program committee. Please elaborate if there are any such issues; these comments will be held in confidence.