| 1 | Oct 13, 2011 5:25 PM | Initially I viewed these scores negatively. When the program chair explained to me that the purpose of these scores was to enable him to monitor the reviewing process more closely, then I understood their purpose, and they didn't really matter to me one way or the other. |
| 2 | Oct 9, 2011 7:12 AM | I think eliminating them would increase quality of the reviews and the review process. |
| 3 | Oct 8, 2011 8:10 PM | I think these fields were very useful to authors. |
| 4 | Oct 7, 2011 9:28 AM | They helped me to clarify and calibrate the overall merit assessment. |
| 5 | Oct 7, 2011 7:52 AM | Each field costs me about a second to think about, and I guess the mark contains a few bytes of information; they express a tendency.
Also, if a fellow reviewer says "Well presented" and the technical part is a mess, that tells me about the informedness about the review.
I believe that everybody knows that those marks don't say much, though. |
| 6 | Oct 7, 2011 7:05 AM | Helped to calibrate score vs. other reviews |
| 7 | Oct 7, 2011 4:57 AM | Sometimes I found them useful, but sometimes I viewed them negatively, because the fields did not always make sense for a particular paper. "Novelty" and "Importance" were especially hard to distill down to a linear order. |
| 8 | Oct 7, 2011 3:41 AM | As a reviewer, It made sure I commented on those aspects in my review. As I didn't feel I could put a score without a comment.
When assessing the papers at the PC meeting it didn't really make much difference. |
| 9 | Oct 7, 2011 2:24 AM | If you are going to have them, it would be better to allow N/A because sometimes they are not relevant. Also, some of them were a bit suggestive. For example a paper might make a massive and important increment over previous work, but because the box is there you are tempted to think "yeah, maybe it's just incremental". |
| 10 | Oct 7, 2011 2:20 AM | The extra fields were not especially useful. Since reviewers wrote a freeform review, it was both easier and more useful to say *why* a paper was (e.g.) unclear in the review, than to just pick a number. |
| 11 | Oct 6, 2011 11:58 PM | I prefer a finer grain overall merit score. |
| 12 | Oct 6, 2011 10:37 PM | If anyone says "I viewed them very negatively" they are an idiot. How can filling out a couple of numbers be a significant negative? |
| 13 | Oct 6, 2011 9:43 PM | They never came up in the PC meeting or discussions of my papers. I don't think they helped me much if at all. |
| 14 | Oct 6, 2011 9:10 PM | I felt reviewers were giving arbitrary values. It's hard to measure these, while one's battling with the primary evaluation of whether the paper is good or not. |