Fred Douglis IBM T.J. Watson Research Center Best Practices for the Care and Feeding of a Program Committee, and Other Thoughts on Conference Organization Fred Douglis IBM T.J. Watson Research Center To replace the title / subtitle with your own: Click on the title block -> select all the text by pressing Ctrl+A -> press Delete key -> type your own text Best Practices 4/9/2019

WOW … It’s WOWCS First reaction: what a great idea! Second reaction: will anyone actually submit? Immediate fear: spend time writing something up, only to find out it’s been canceled Final reaction: duty I’ve chaired USENIX’98, USITS’99, and a few other conferences Most went well but there were some issues End goal: guidebook (wiki) for program/general chairs https://wiki.usenix.org/bin/view/Main/Conference/CollectedWisdom A pot file is a Design Template file, which provides you the “look” of the presentation You apply a pot file by opening the Task Pane with View > Task Pane and select Slide Design – Design Templates. Click on the word Browse… at bottom of Task Pane and navigate to where you stored BlueOnyx Deluxe.pot (black background) or BluePearl Deluxe.pot (white background) and click on Apply. You can switch between black and white background by navigating to that pot file and click on Apply. Another easier way to switch background is by changing color scheme. Opening the Task Pane, select Slide Design – Color Schemes and click on one of the two schemes. All your existing content (including Business Unit or Product Names) will be switched without any modification to color or wording. Start with Blank Presentation, then switch to the desired Design Template Start a new presentation as Blank Presentation You can switch to Blue Onyx Deluxe.pot by opening the Task Pane with View > Task Pane and select Slide Design – Design Templates. Click on the word Browse… at bottom of Task Pane and navigate to where you stored BlueOnyx Deluxe.pot (black background) and click on Apply. Your existing content will take on Blue Onyx’s black background, and previous black text will turn to white. You should add your Business Unit or Product Name by modifying it on the Slide Master You switch to the Slide Master view by View > Master > Slide Master. Click on the Title Page thumbnail icon on the left, and click on the Business Unit or Product Name field to modify it. Click on the Bullet List Page thumbnail icon on the left, and click on the Business Unit or Product Name field to modify it. Click on Close Master View button on the floating Master View Toolbar You can turn on the optional date and footer fields by View > Header and Footer Suggested footer on all pages including Title Page: Presentation Title | Confidential Date and time field can be fixed, or Update automatically. It appears to the right of the footer. Slide number field can be turned on as well. It appears to the left of the footer. Best Practices 4/9/2019

Wiki Goals Two types of info eventually Living document Allow discussion Two types of info eventually Guide for new program chairs or conference organizers Review management software Best practices Ideas for new ways to do things How well do rebuttals work? Ratings? Bidding? Best Practices 4/9/2019

Problem 1: Bad Reviewers Not everyone is a good PC member Some never do anything Some do really crummy reviews Some are too argumentative, dominate the rest of the PC, or have other personality issues How to find who is a known bad reviewer? Past experience Word of mouth… but people tend not to know to ask How to expand word of mouth? Reviewer database? Best Practices 4/9/2019

Problem 2: Inexperienced Reviewers Always want to bring some “new blood” into a PC Search for repeat authors who haven’t served (need a script to scour Google Scholar for this) Caveat: not everyone knows what to expect and may not be cut out for it. Don’t take too many people you don’t know if you can help it Tips for ensuring the best PC Set expectations early (# reviews, timing, … no surprises!) Have multiple deadlines: force people to miss early rather than all at once just before the decisions Help with calibration (average scores of reviewers compared to the scores of their peers on same papers) Best Practices 4/9/2019

Problem 3: PC Composition Avoid inbreeding, or the appearance of it Overlap from year to year Institutional overlap There are enough people who will have published at an established conference that a chair should draw on them rather than outside the conference community Best fit Reward participation Avoid problems with calibration Best Practices 4/9/2019

PC Bells and Whistles Options for running a PC Rebuttals (Seem to be a nice idea, but add lead time) Reviewer ratings (hard to calibrate; don’t want to insult) Best Practices 4/9/2019

Running a Conference Sponsorship Setting expectations Scheduling Some institutions make it a pleasure to run a conference, and some make it much harder Doing it alone has risks (e.g., liability): who “owns” the conference risks? Setting expectations It’s hard to know just how many submissions a new conference will get: better to be swamped (and add PC members) than miss all your targets Publicity is amazingly critical (who knew?) Scheduling Watch for conflicts and hope the others watch too Consider the impact of rejected papers from one conference being routed to another. (Feed reviews?) Best Practices 4/9/2019

Backup Best Practices 4/9/2019

Problem 4: (Self)plagiarism Simultaneous submission seems to be a big problem Least publishable units True plagiarism We need a better mechanism for detecting Serendipity doesn’t cut it Best Practices 4/9/2019

Concrete Proposals/Challenges Plagiarism Neutral agent to collect papers and analyze for overlap Get multiple organizations to buy in Reviewer database Neutral agent to collect feedback, and if someone gets multiple negative reports, somehow blacklist See Internet Computing columns Sept-Oct/Nov-Dec 2007 Best Practices 4/9/2019