| 2. What was the outcome of your submission(s) to POPL? | |||
|---|---|---|---|
| answered question | 276 | ||
| skipped question | 0 | ||
| Response Percent | Response Count | ||
| Accepted |  | 29.7% | 82 |
| Some accepted and some rejected/withdrawn (only for multiple submissions) | | 6.2% | 17 |
| Rejected or withdrawn | | 60.5% | 167 |
| Prefer not to say | | 3.6% | 10 |
| 3. Were you on the POPL'12 program committee (PC) or extended review committee (ERC) ? | |||
|---|---|---|---|
| answered question | 276 | ||
| skipped question | 0 | ||
| Response Percent | Response Count | ||
| I was on the PC | | 4.3% | 12 |
| I was on the ERC | | 8.3% | 23 |
| I was not on either the PC or the ERC | | 87.3% | 241 |
| 4. Please indicate your opinion of which choice (among only these two) is best: single-blind reviewing (SBR) as is typically employed by POPL, or double-blind reviewing (DBR) as we implemented it this year (as per the POPL'12 CFP, and the FAQ at http://www.cs.umd.edu/~mwh/dbr-faq.html). Your answer should reflect your perception of the best choice on balance, based on which process you think is overall the most fair, most accurate, most useful, etc. | |||
|---|---|---|---|
| answered question | 246 | ||
| skipped question | 30 | ||
| Response Percent | Response Count | ||
| Traditional single-blind reviewing | | 29.7% | 73 |
| Double-blind reviewing as per POPL'12 this year | | 70.3% | 173 |
| Optionally provide reasons or qualifications for your choice; experiences from this year most welcomeview | 68 | ||
| 5. How has your opinion about DBR changed as a result of your experience with POPL'12? | |||
|---|---|---|---|
| answered question | 214 | ||
| skipped question | 62 | ||
| Response Percent | Response Count | ||
| My opinion of it improved a lot | | 11.2% | 24 |
| My opinion of it improved a little | | 58.4% | 125 |
| I think less of it than before | | 27.1% | 58 |
| I think much less of it than before | | 3.3% | 7 |
| Feel free to expand on your answerview | 61 | ||
| 6. How would you characterize the POPL'12 requirements for blinding your submission? | |||
|---|---|---|---|
| answered question | 229 | ||
| skipped question | 47 | ||
| Response Percent | Response Count | ||
| They were easy to follow | | 76.0% | 174 |
| They were mostly easy, but I was unsure about certain corner cases that applied to me | | 21.8% | 50 |
| They were confusing; I often needed more elaboration (or fewer rules) | | 2.2% | 5 |
| Elaboration or other comments/suggestionsview | 17 | ||
| 7. Describe the changes you had to make to your submission, or extra work you performed, for this year's POPL compared to what you would have done if POPL had used SBR (check all that apply): | |||
|---|---|---|---|
| answered question | 178 | ||
| skipped question | 98 | ||
| Response Percent | Response Count | ||
| changed self-references to the third person | | 89.3% | 159 |
| anonymized citations for papers simultaneously under review | | 20.8% | 37 |
| removed qualitative judgments that appeal to your past experience (please describe) | | 12.4% | 22 |
| omitted text, examples, or references that you felt might have revealed your identity (please describe) | | 24.7% | 44 |
| Other (or elaboration on the above)view | 45 | ||
| 8. How would you characterize the effect of the above changes on the quality of your manuscript? | |||
|---|---|---|---|
| answered question | 226 | ||
| skipped question | 50 | ||
| Response Percent | Response Count | ||
| They improved it substantially | | 0.4% | 1 |
| They improved it slightly | | 0.9% | 2 |
| They had basically no effect | | 78.8% | 178 |
| They hurt it slightly | | 18.1% | 41 |
| They hurt it substantially | | 1.8% | 4 |
| Please elaborate on your answer (esp. the two extremes)view | 13 | ||
| 9. Consider how you felt while your work was under review by the POPL'12 committee. Compared to how you would feel when using a typical SBR process, to what extent did the double-blind process make you feel uneasy or awkward about disseminating your work, discussing your work with colleagues, or giving talks at institutions where a PC or ERC member might learn your identity? | |||
|---|---|---|---|
| answered question | 228 | ||
| skipped question | 48 | ||
| Response Percent | Response Count | ||
| I felt quite uneasy and/or awkward | | 6.1% | 14 |
| I felt a little uneasy and/or awkward | | 28.1% | 64 |
| I felt no differently than I would for a SBR process | | 65.8% | 150 |
| 10. How did you change your actions as a result of adhering to the requirements given on the FAQ (http://www.cs.umd.edu/~mwh/dbr-faq.html) ? Mark those activities that you specifically avoided but would have done if DBR had not been place. | |||
|---|---|---|---|
| answered question | 101 | ||
| skipped question | 175 | ||
| Response Percent | Response Count | ||
| did not send paper directly to PC/ERC members for comment | | 52.5% | 53 |
| did not announce draft of paper or related artifact on mailing list, forum, etc. | | 76.2% | 77 |
| did not offer to give a talk at an institution at which a PC/ERC member is located | | 38.6% | 39 |
| Other (or elaboration)view | 24 | ||
| 11. Please share any other thoughts you have about the DBR-style review process for POPL this year; constructive criticism is particularly welcome! | ||
|---|---|---|
| answered question | 66 | |
| skipped question | 210 | |
| Response Count | ||
| view | 66 | |