1. INFOCOM 2008 TPC Informational Meeting Anchorage, Alaska May 9, 2007 TPC Co-Chairs Jennifer Hou (UIUC) Shivkumar Kalyanaraman (RPI) Krishna Sivalingam (Univ of Maryland, Baltimore County) EDAS Chair Byrav Ramamurthy, Univ of Nebraska, Lincoln

2. Thanks to… 265+ TPC members 31 Area TPC Chairs Byrav Ramamurthy, EDAS chair KK Ramakrishnan (AT&T) & Byrav: mini-conference co-chairs Henning Schulzrinne for help with EDAS INFOCOM Standing Committee and in particular, Harvey Freeman Several former TPC co-chairs and members of the community for suggestions on improving the process

3. High-Level Objectives & Changes Combat random selection of borderline papers: mechanisms for sounder decisions Provide better feedback to authors: quality/quantity Summary of changes/mechanisms: –Better matching of papers to reviewers (fine-grained topics & methodology: simln/expt/modeling) –Early rejects by TPC-leads: Focus reviewing attention –TPC leads: soft oversight on review quality/quantity during discussion phase –Area chair process: Handle borderline papers & non-authoritative reviews Recommend decisions on accept/reject/discuss –TPC meeting grouping: TPC members will be in groups where their reviewed papers are discussed (as much as possible)

4. Overview: Matching Papers  TPC members Observations: –Modeling papers better reviewed by modeling community –Authors select too many topics now. Primary Methodology also mentioned in topic –SIM(ulation), MOD(eling) or EXP(erimental) –Tradeoff: many micro-topics Authors specify at most TWO topics per paper + ONE methodology TPC members select several topics of interest + methodology.

5. Overview: TPC-lead Early Rejects & Soft Oversight TPC lead expanded responsibilities: –Early reject recommendations up to 2 out of TPC-lead papers. Need to do one full review. –TPC discussion phase: soft oversight over review quality Ensure scores are coherent with written reviews, Followup with reviewers on short reviews, inconsistencies Don’t reject papers for tiny flaws, marginal issues. Flag paper for ATPC attention if majority of reviews are not authoritative reviews or for papers with genuine disagreements. Since everyone is a TPC-lead for some papers, we hope it will reinforce responsibility for review quality and lead to better reviews/feedback to authors.

6. Overview of ATPC Chairs in Review Process Paper ATPC Chair TPC Reviewers(2) TPC Lead Reviews; TPC Summary; Recommendation TPC Meeting ATPC Recommendation Addl Review Backup TPC Co-chairs Early Reject

7. Details: Review Process, Phase I Due DateAction Item June 1TPC Members: Topic Selection / Conflict of Interest Update July 2, 11:59PM PDTAuthors: Papers submissions complete July 8TPC Members: Claim papers August 1TPC Co-Chairs: Assign 1 TPC Lead and 2 TPC reviewers per paper (papers likely to be assigned earlier) August 15TPC Members: Recommend 0-2 Early Rejects from their papers in TPC-Lead category Sep 8TPC Members: Complete Reviews of all Papers Sep 22TPC Members: Complete On-line Discussion and TPC Lead Summaries/Recommendation: Accept, Reject, Discuss, Addl. Reviews

8. Details: Review Process, Phase II Due DateAction Item Sep. 22ATPC Chairs: Review top 50% of papers in assigned ATPC Group and identify papers needing additional review(s) Sep. 22TPC Co-Chairs: Review bottom 50% of papers in each ATPC Group to verify reject recommendation Oct. 8ATPC Chairs: Ensure additional review(s) are completed Oct. 15ATPC Chairs: Recommend paper decisions to TPC Co-Chairs: Accept, Reject, Discuss Oct. 27TPC Meeting: Washington, DC Nov. 1 st weekTPC Co-Chairs: Author Notifications

9. Details: Submission Guidelines (June 25) Paper registration: Authors provide a title, abstract and keywords to register their paper. –Authors select ONE Review topic group that best fits the subject matter of the paper. e.g. 5.4 Mobile Ad Hoc Networks –Authors select at least ONE and at most TWO topics from the list of possible paper topics e.g. Ad Hoc Networks - Routing and Multicasting - SIM –Authors indicate your agreement and compliance with the INFOCOM 2008 Dual Submission policy. (July 2) Paper submission: Authors upload the manuscript (up to 9 pages, PDF format) –Authors can also list up to THREE reviewers (TPC members or others) who they DO NOT prefer review their manuscript.

10. Details: Regular Review vs. Lead TPC Review/Summary Each TPC member will be: assigned around 20 papers of which he/she will be TPC lead for about 7-8 papers Of these 20 papers: –TPC member must read/review the TPC Lead papers: NO DELEGATION –Other half of regular TPC review papers (6): The TPC member assigned is responsible for the review and discussion. If graduate students/post-docs *help* with the review, the TPC member is responsible for overseeing the quality, coherence of the review and finally ENTERING the review into EDAS. The TPC member must also actively participate in the post-review on-line discussion phase. –One-half of regular TPC review papers (6): Delegate to qualified researcher via EDAS, but *NOT* graduate students; Delegation must be done within 2 weeks of papers assigned to the TPC member. You must ensure that delegated reviews are done on time!

11. Details: Early Reject After a high-level reading of the TPC Lead Papers, identify 0-2 papers for Early Reject: –Paper should clearly be at the bottom of the list of the TPC Lead papers assigned to you –The papers should not be worthy of 3 full reviews, and highly likely to earn scores of 1,1,1 if fully reviewed. –Examples: sloppy writing, presentation, totally confused concepts, very minor contribution, etc. Papers with reasonable concepts etc. deserve full review/feedback even if the TPC lead may not like the concept. –TPC member MUST be absolutely sure that a given paper can be rejected without any controversy. – It is perfectly fine if you do not choose any paper for early reject, from your assigned list. Make your recommendations by August 15th, 2007: Enter a detailed review TPC Co-Chairs may occasionally consider a paper for full review despite the TPC lead's recommendation for early reject

12. Regular Review vs. Lead TPC Review/Summary: Discussion Phase Actively participate in online discussion phase: –Consider objections raised by others and either concur or correct/disagree –Avoid the impulse to push paper to discuss category straightaway! We are time-constrained in TPC meeting –TPC-lead provides soft-oversight over quantity/quality/consistency of written review & scores Once the discussion phase is over, TPC lead provide a summary of the discussions in the form of a TPC review and select one of: –Accept –Discuss at TPC meeting –Needs Additional Review and –Reject. TPC summary will not be made available to the authors

13. Details: Area TPC Chairs ATPC Chairs can monitor reviews and discussions from Sep. 8 onwards –Can help identify controversial/borderline papers early on –Please do not participate during TPC online discussion phase On Sep. 22 nd, please identify papers needing additional review –You may enter reviews yourself –You may ask qualified RELIABLE reviewers –You may utilize a subset of TPC members (backup TPC members), who are assigned to help during this phase By Oct 15, make recommendation to TPC Co-Chairs Accept, Discuss at TPC meeting, Reject Your presence is important during TPC meeting to lead the discussion groups

14. Area Chairs: Guidelines (Excerpts) The TPC Co-Chairs will immediately suggest a filter threshold for clear rejects and the remaining papers need the attention of the Area TPC chairs in a fairly short period (3 weeks). –Assuming that there are 1500 submissions and 30 ATPC Chairs, there will be 50 papers to handle per ATPC member. –Of this, the TPC Co-Chairs will review the bottom 50% of papers in each group (i.e. approx 25 papers) and reject them (for most of them). –We anticipate that each Area TPC chair will have to actively deal with 25-30 papers. –Some will be clear accepts with the area chairs will quickly recommend/confirm, and most papers will be in the Discuss/Needs Extra Review categories (but could also include some "weaker" accept/rejects with conflicting scores as well). –Based upon their expertise, the Area TPC Chairs could choose to reject some more papers without an extra review. –About 15-20 papers may require an extra review to help the decision making. The Area TPC chairs can use the Backup TPC members (who have not been given assignments in the beginning) to get one extra review per paper. –Area TPC Chairs might also find it very useful to have a list of pre- identified RELIABLE REVIEWERS (who can handle 1-2 papers at short notice) and notify them in Aug/Sept. 2007 that their services might be needed in October. http://cse.unl.edu/~byrav/INFOCOM2008/area-tpc-chair-guidelines.html

15. ATPC Guidelines: Excerpts (Contd) Based upon the extra review, the Area TPC chair can reject (if the review confirms/clarifies other reject recommendations), recommend accept (if the consensus is very strong) or recommend discussion at the TPC meeting. –We encourage you to reject decisively based upon the extra reviews to reduce the decision load for the TPC meeting. –While you have authority to recommend ACCEPTs, we urge some caution because of the limited number of total accepts and the need for the TPC meeting to weigh in if papers do not have clear consensus for accept. As in the INFOCOM 2007 TPC meeting, –our objective is to have about 300 total papers in the TPC meeting with about a 40-50% accept rate at the meeting to keep the meeting positively focused. –The Area TPC process would have provided extra reviews to help guide the discussion process at the meeting. –We expect between 10-15 of your 30 papers may make it to the meeting, and are those that deserve the 40-50% probability for acceptance. ATPC members and the TPC members who reviewed the paper will be matched to the same groups at the TPC meet (to the best extent possible)

16. Questions? Email: inftpc08@cse.unl.eduinftpc08@cse.unl.edu Individual Emails: –jhou@cs.uiuc.edujhou@cs.uiuc.edu –shivkuma@ecse.rpi.edushivkuma@ecse.rpi.edu –krishna@cs.umbc.edukrishna@cs.umbc.edu –byrav@cse.unl.edubyrav@cse.unl.edu URLs describing review process: –http://cse.unl.edu/~byrav/INFOCOM2008/http://cse.unl.edu/~byrav/INFOCOM2008/ Guidelines for Area TPC Chairs, TPC members, Reviewers

17. Review Quality High quality reviews is by far the most important contribution of a TPC member –Discussions on TCCC and elsewhere show growing discontent with INFOCOM reviews/feedback and the decision process –“Do unto others as you would like done to you” – Write reviews of the quality that you would like to receive for your papers Basic Quantity/Quality: Reviews shorter than 100 words will generate automated EDAS feedback –Do not cut-and-paste same information in different categories If any of the papers are reviewed with the help of students, the TPC member must provide oversight and clarity and personally enter the reviews Ensure coherence between numerical scores and review content –If you want your reviews to matter in TPC discussions, please enter substantive, high quality reviews. –TPC leads also provide soft oversight in discussion phase

18. Backup TPC members No regular assignments Help ATPC chairs with authoritative reviews during short period Help with any missing reviews