1. The Blind Leading the Blind
Prof. Short*
Dr. Tall*
* Names obfuscated and affiliations
omitted to preserve anonymity

2. Outline The five best features of double-blind reviewing
Beyond double-blind

3. Feature #1
Enables useful feedback on half-baked papers, without fear of embarrassment
(keeping those under-worked PC members busy)

4. Feature #2
Slows the advancement of science to a manageable pace, by eliminating rapid dissemination of results

5. Feature #3
Allows job-seeking PhD students to allude to spectacular new results, which unfortunately they can’t talk about
(regardless of whether the results actually exist)

6. Feature #4
Discourages those annoying high-impact projects with recognizable names and many-author papers that build on one another

7. Feature #5 Facilitates “flow” of ideas from authors to reviewers
(without the irritating requirement of attribution)

8. But…
Double-blind doesn’t go nearly far enough

9. Problem #1
Senior reviewers can intimidate the junior reviewers of a paper during discussions

10. Problem #1
Senior reviewers can intimidate the junior reviewers of a paper during discussions
SOLUTION: Triple-Blind
Reviewers don’t know who the other reviewers are

11. Problem #1
Senior reviewers can intimidate the junior reviewers of a paper during discussions
SOLUTION: Triple-Blind
Reviewers don’t know who the other reviewers are
This one is real!

12. Problem #2
Authors of high-impact papers become more famous than authors of insignificant papers

13. Problem #2
Authors of high-impact papers become more famous than authors of insignificant papers
SOLUTION: Quadruple-Blind
Authors of published papers are anonymous

14. Problem #2
Authors of high-impact papers become more famous than authors of insignificant papers
SOLUTION: Quadruple-Blind
Authors of published papers are anonymous
Someone (perhaps Jim Gray...) was 20 years ahead of his or her time with the 1985 “Anon et al.” benchmarking paper

15. Problem #3
System is biased in favor of authors who give great talks about their results

16. Problem #3
System is biased in favor of authors who give great talks about their results
SOLUTION: Quintuple-Blind
PC chair gives all the talks

17. Problem #4
Famous researchers decline to serve on PCs for second-tier conferences

18. Problem #4
Famous researchers decline to serve on PCs for second-tier conferences
SOLUTION: Sextuple-Blind
Conferences are anonymous — PC members don’t know what conference they’re agreeing to review for

19. Problem #5
Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues

20. Problem #5
Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues
SOLUTION: Septuple-Blind
Conference submissions are picked randomly from a global pool

21. Acknowledgements Thanks to … from whom we “borrowed” some of these
An anonymous west-coast professor with a photography habit
An anonymous Midwest professor with the same first name as his (or her!) advisor
from whom we “borrowed” some of these
ideas (when they weren’t looking)