1. Customer Success is Our Mission MILCOM 2008 Reviewer Guidelines Rev B 8 July 2008

2. Customer Success is Our Mission 2 Paper Selection for MILCOM MILCOM presents papers of interest to the military communications community which includes persons with diverse interests such as –Latest advances in theoretical communications applicable to the military –Latest technical advances in components and systems –Detailed capabilities of current and proposed military communications systems –Concept of operations for military communications systems Reviewers (and Session Chairs) need to consider the papers intended audience and adjust review standards appropriately E.g. Papers on Software Communications Architecture (SCA) are of significant interest to the community given the importance of SCA to the JTRS programs

3. Customer Success is Our Mission 3 Reviewer Selection Sources of candidate reviewers –WAMS reviewer database Includes 2007 reviewers MILCOM 2007 Reviewers by Session report available –COMSOC reviewers available with subject matter experts (SMEs) in most Comm areas Provide Unclassified Program Chair (UPC) with requirements –Track and Session Chairs know SMEs within their own organizations Use TC telecons to advertise need for reviewers with specific expertise Guidelines for reviewer selection –Draft paper author prohibited from reviewing her/his own paper! Avoid conflicts of interest in general (e.g. dont assign all reviewers from the same organization to a given paper) –In addition Session organizer should not handle the review of his/her own paper! –Ideal reviewers have written papers on the same topic as the draft paper to be reviewed –SMEs in the draft paper topic –Accept senior graduate students as reviewers. Limit to 1 graduate student review out of every 3 reviews of a particular paper Reviewer qualifications –Good judgment in reviewing papers (understands the big picture and topics of interest to the MILCOM audience) –Track record of submitting reviews on time! –Track record of providing good review comments for both Session Chair and author

4. Customer Success is Our Mission 4 Writing Reviews WAMS provides two types of reviews –Review comments for author to improve paper –Review comments for Session Chair providing critique of paper and rationale for acceptance/rejection Purpose of reviews –Provide rationale for accept/reject decision (help Session Chairs make final accept/reject decision when reviewers disagree on accept/reject decision) –Provide feedback to authors to improve papers Reviewers should not indicate accept/reject decision in comments to authors Comments to authors should be of sufficient detail so that author is not surprised by subsequent rejection notification WAMS review evaluation form –Requests numerical score from 5 -> 1 in 4 areas (see page 6 for definition of numerical scores) Technical Contribution (rate content based on technical strength) Advancing Technologies/State-of-the-art (novelty or contribution to military comm) Relevance and Timeliness (importance to military comm applications) Quality of Presentation (readability of paper) –WAMS provides a sum of the numerical 5 -> 1 score in the four areas Accept/reject score can be Session specific In general expect acceptance to require a score > 12 (what do the TCs think of a minimum score?) –Requests accept/reject recommendation –Comments to authors (be courteous but accurate) WAMS requires a minimum of 200 characters to force reviewers to provide feedback Recommend reviewers provide (for author) –Brief overview of paper scope –Summary of contributions and positive aspects –Weaknesses and problems in paper –Detailed comments for authors Note that weaknesses and problems in paper should justify a rejection decision –Comments to Session Chair/Organizer Justification of overall rating and accept/reject decision.

5. Customer Success is Our Mission 5 Writing Reviews (cont) Further recommendations for paper reviews –SMEs tend to be very critical of papers in their subject while more liberal regarding papers outside their area of expertise (beware!) –Distinguish between correctable errors and flaws If reviewer believes errors must be corrected before publication then so state in the review –Keep the MILCOM program scope in mind during review Some papers may be outside the MILCOM topic areas and shouldnt be presented at MILCOM (but perhaps at a different conference) –Ensure that papers do not exceed the MILCOM page limit (10) Note that authors are required to pay an additional $330 per page for pages 8 – 10 –Avoid middle category (3) ratings as much as possible since such ratings dont help the selection process Historically the MILCOM draft paper accept ratio has ranged from 50% - 60% –Reviewers should reject papers if, in their opinion, the work is below average Note that it is possible that some reviewers will rate all papers above average since they were assigned above average papers! –Note that not all Sessions will achieve the average acceptance ratio Track Chairs need to review Sessions where accept ratio differs significantly from average (up or down) At the Track level the law of large numbers must prevail!

6. Customer Success is Our Mission 6 Definition of Review Numerical Score Definition of numerical score –5: Excellent - Outstanding technical content and excellent presentation –4: Good - Solid work of notable importance and good presentation –3: Acceptable - Valid work but limited contribution and average presentation. –2: Marginal - Simple contribution with some flaws. –1: Unacceptable - Questionable work with severe flaws.

7. Customer Success is Our Mission 7 Use of WAMS for Review Process How to assign reviewer from WAMS –See attached PDF file from S. Kota on this topic How to submit reviews in WAMS –See attached PDF file from S. Kota on this topic