The Blind Leading the Blind Prof. Short* Dr. Tall* * Names obfuscated and affiliations omitted to preserve anonymity

Outline The five best features of double-blind reviewing Beyond double-blind

Feature #1 Enables useful feedback on half-baked papers, without fear of embarrassment (keeping those under-worked PC members busy)

Feature #2 Slows the advancement of science to a manageable pace, by eliminating rapid dissemination of results

Feature #3 Allows job-seeking PhD students to allude to spectacular new results, which unfortunately they can’t talk about (regardless of whether the results actually exist)

Feature #4 Discourages those annoying high-impact projects with recognizable names and many-author papers that build on one another

Feature #5 Facilitates “flow” of ideas from authors to reviewers (without the irritating requirement of attribution)

But… Double-blind doesn’t go nearly far enough

Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions

Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions SOLUTION: Triple-Blind Reviewers don’t know who the other reviewers are

Problem #1 Senior reviewers can intimidate the junior reviewers of a paper during discussions SOLUTION: Triple-Blind Reviewers don’t know who the other reviewers are This one is real!

Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers

Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers SOLUTION: Quadruple-Blind Authors of published papers are anonymous

Problem #2 Authors of high-impact papers become more famous than authors of insignificant papers SOLUTION: Quadruple-Blind Authors of published papers are anonymous Someone (perhaps Jim Gray...) was 20 years ahead of his or her time with the 1985 “Anon et al.” benchmarking paper

Problem #3 System is biased in favor of authors who give great talks about their results

Problem #3 System is biased in favor of authors who give great talks about their results SOLUTION: Quintuple-Blind PC chair gives all the talks

Problem #4 Famous researchers decline to serve on PCs for second-tier conferences

Problem #4 Famous researchers decline to serve on PCs for second-tier conferences SOLUTION: Sextuple-Blind Conferences are anonymous — PC members don’t know what conference they’re agreeing to review for

Problem #5 Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues

Problem #5 Researchers insist on sending their best work to the best conferences, which is unfair to second-rate venues SOLUTION: Septuple-Blind Conference submissions are picked randomly from a global pool

Acknowledgements Thanks to … from whom we “borrowed” some of these An anonymous west-coast professor with a photography habit An anonymous Midwest professor with the same first name as his (or her!) advisor from whom we “borrowed” some of these ideas (when they weren’t looking)