This page is about the reviews for the ESA 2018 Track B PC.
Contents
Text of each review
Each review should provide the following information:
1. A short summary of the main contribution(s) of the submission in the words of the reviewer 2. An itemized list of the strength and weaknesses of the submission 2.1 The strengths should be numbered (S1), (S2), ... 2.2 The weaknesses should be numbered (W1), (W2), ... 3. More detailed explanations of the strenghts and weaknesses (if necessary or useful) 4. Comments to the authors for improving the paper (if applicable)
Score of each review
Each review should provide one of the following scores.
Score |
Verdict |
Behavior during discussion |
+2 (accept) |
No major weaknesses |
I would champion this paper and fight against rejection |
+1 (weak accept) |
Significant weaknesses, but nothing fatal |
I would support this paper, but not fight against rejection |
0 (borderline) |
Hovering between +1 and −1 |
Not sure yet about the severity of the weaknesses / the threshold for ESA |
−1 (weak reject) |
Significant weaknesses, but nothing fatal |
I am not supporting this paper, but would also not fight against acceptance |
−2 (reject) |
Major weaknesses |
I am opposing this paper and fight against acceptance |
Remark 1: Some conference also have +3 (strong accept) and −3 (strong reject). Experience shows that they are of little use for deciding on the set of accepted papers for a moderate number of submissions, as in ESA Track B (around 50).
Remark 2: Some conferences disallow the borderline score of 0, to enforce a clear opininion on the reviewer. In the discussion phase, we indeed ask revievers to commit to one of the other scores. But for the first review, we think it makes sense to allow this score, because it reflects one of the typical sentiments about a paper at this stage of the reviewing process, as expressed by the hovering between +1 and −1 in the table above.
Remark 2: Reviewers might not be fully aware yet of their behavior during the discussion phase for various reasons (for example: not sure about some aspects of the paper, not sure about the nature of the threshold for ESA Track B, general inexperience in reviewing). This can make choosing the right score difficult. But this is exactly one of the tasks of the discussion phase (described below): to bring the final scores (and reviews) closer to what they were supposed to reflect.