IEEE-SA Sponsor Ballot: Comment Resolution PE/Surge Protection Devices Committee May 24th, 2017 St Louis, MO

Agenda Ballot Comment Resolution What it is IEEE-SA Operations Manual Do’s and Don’ts Best Practices RevCom Comments Questions

Comment Resolution After the Sponsor ballot has closed, the Working Group will need to respond to the commenters. This can often be a daunting task for ballots that have a large number of comments. Know the process, what to avoid, what to prepare, etc.

IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts) Comments addressing grammar, punctuation, and style, whether attached to an affirmative or a negative vote, may be referred to the publications editor for consideration during preparation for publication. It should be borne in mind that documents are professionally edited prior to publication. After balloting closes, voters who have voted “Disapprove”, may “flip” their vote to either “Approve” or “Abstain” at any time before the next recirculation or the RevCom meeting.    

IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts) Changes may be made in the document to resolve negative votes that are accompanied by comments or for other reasons. All substantive changes made since the last balloted draft shall be identified and recirculated. All unresolved negative votes with comments shall be recirculated. During a recirculation ballot, balloting group members shall have an opportunity to change their previously cast ballots. A change to "disapprove," which is submitted with comments, shall be based only on; changed portions of the balloted document, clauses affected by the changes, or portions of the balloted document that are the subject of the unresolved negative votes.

IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts) If a change to "disapprove" is based solely on comments concerning previously approved portions of the balloted document, the balloter shall be informed that the comments are not based on the changed portion of the balloted document and, therefore, those comments may not be addressed in the current ballot and may be considered for a future revision of the standard or may be considered out of scope. If the balloter does not agree to change the negative ballot, the ballot shall be recorded with an unresolved negative without comment. Additional recirculation ballots shall be required if negative votes with new comments within the scope of the recirculation result. Proposed standards receiving a significant number of unresolved negative votes should be considered by the Sponsor for trial-use (see Standards Operations Manual, Section 5.7).

IEEE-SA Ops. Manual Clause 5.4.3.3 (Excerpts) The Sponsor Ballot is the process whereby the Balloting group is trying to reach consensus on draft standard. Consensus means agreement among the majority. According to IEEE rules, consensus is defined as a minimum 75% return of ballots from the balloting group, and a 75% approval rate from that 75% return group. The Sponsor shall make a reasonable attempt to resolve all comments, objections, and negative votes that are accompanied by comments. Comments that advocate changes in the document, whether technical or editorial, may be accepted, revised, or rejected. All comments from the Sponsor Ballot must be considered and responded to by the WG or Ballot Resolution Group (BRC), this process may change the draft which may in result in changed votes. Negative comments typically explain any difficulties the balloter has with the current document and offer precise wording for changes that would turn Disapprove Votes to Approve Votes. In many cases, the balloter may offer vague solutions or even no solution at all. At this point, the WG or BRC should examine the problem to see if they can resolve it on their own, or they may discuss the situation with the balloter and solicit more precise language. If a negative vote comes without comments, it cannot be resolved.

Resolution Statuses Defined: Accepted – The Ballot resolution committee (BRG) or Comment resolution group (CRG) agreed exactly with comment and change proposed by the commenter. The proposed changes contain sufficient detail so that voters can understand the specific changes that satisfy the commenter and the editor can make the change Response: The disposition detail field should be left blank Revised –If the CRC agrees with the comment, but disagree with specific changes the Disposition Status should be "revised" The CRG disagrees with all or part of the specific details in the proposed change in the comment, the proposed change does not contain sufficient detail for the CRG to understand the changes; changes made by the CRG contain additions or modifications to what was proposed, Response: The disposition detail field should contain sufficient detail so that voters understand the specific changes determined by the CRG or an editor can change 9/19/2018

Resolution Statuses Defined: Rejected - the CRG disagrees with the comment or the comment is out of scope. The proposed change in the comment does not contain sufficient detail so that the CRG can understand the specific changes the CRG cannot come to a consensus to make changes necessary to address the comment the comment is in support of an unsatisfied previous comment associated with a disapprove vote and does not provide substantive additional rationale the comment includes an attachment that does not meet the criteria indicated by the myBallot system; that the CRG cannot address as a single issue; or that does not relate to a specific line, paragraph, figure, or equation in the balloted draft or the commenter has indicated to the CRG chair that they wish to withdraw the comment Response: The disposition detail field should explain why the comment is rejected: an explanation of why the CRG disagrees with the comment, a statement that the comment is out of scope, and the rationale, a statement that the proposed change in the comment does not contain sufficient detail so that the CRG can understand the specific changes

Do’s and Don’ts Don’t make, or promise to make, any post-ballot changes to the draft.   If you do RevCom would require a recirculation. RevCom does review the last balloted draft Do reject an editorial comment and defer it to IEEE-Editorial for review (not for change) You cannot make the promise that IEEE-Editorial will indeed make the change, just defer for review for change. Example Comment: Resize Figure 1 to fit the page better Don’t Use: Accepted. Will resize the figure. Do Use: Rejected. Defer this change to IEEE-Editorial staff for review and change during final publication, use suggested response: “Will let Editorial staff determine if changes can possibly be made during publication.”

Do’s and Don’ts For Revised or Rejected status don’t have any empty disposition status or detail columns All comments need to be shown consideration If “accepted” a leave the disposition detail column empty Do have sufficient detail explaining the reason for rejection or the revised comment Example of rejected comments: Don’t use: Rejected. More discussion required. Where were the sensor experts when we requested assistance in writing these requirements?" Do Use: Rejected: Commenter did not provide sufficient details or any proposed changes for the CRG to make changes to the draft.

Do’s and Don’ts Don’t have an outstanding negative ballot with comment(s) and submit to RevCom without addressing the comment(s) Do email the balloter a reason for rejecting the comment and advise he/she that you are submitting to RevCom. Do include this email to RevCom in your submission

Do’s and Don’ts Don’t cross reference “Disagree – see resolution of comment 1234” Do cut and paste the comment resolution from the cited comment and then add “ (same comment resolution as comment 1234)”

Do’s and Don’t’s Don’t reference an external document “Disagree – see 11-09/9876r1” Not an appropriate response Do copy information from document 11-09/9876r1 paste the information from the document in the disposition detail

Best Practices The WG Chair can form a CRG to resolve the comments Document the formation and the scope of the Adhoc in the WG meeting minutes Resolve the comments using the disposition statuses ; accepted, rejected, revised Can split work and assign roles to expedite process Ensure accepted changes are incorporated into the Draft Note substantive changes in the cover letter (if needed) Note substantive changes in the Draft offered for recirculation Deleted material in the Draft should be indicated using strikethrough. Any color may be used for the strikethrough. Any color is acceptable to indicate changes. Location of deleted and new material should be indicated using a change bar in the margin

Submission to RevCom Final approval of a standard is achieved by submitting the document and supporting material to RevCom (Review Committee), which issues a recommendation to the IEEE-SA Standards Board Once a project is submitted to RevCom the Working Group Chair is responsible to responding to any comments/questions raised by the RevCom Members All Dialogue and Responses are done via myProject WG Chairs will receive automatic notification email from myProject that a comment has been made Don’t reply to the email Do log into myProject and post a response Don’t delay in responding to RevCom, the longer you wait, the likelihood of additional comments increases.

IEEE-SA RevCom Comment Resolution Preparation Guidelines RevCom will review all the rounds of balloting, the comments, disposition status and detail along with the drafts. RevCom will base their review, comments, and recommendations on: RevCom Conventions RevCom Comment Resolution Preparation Guidelines The Guideline is intended to provide assistance to the Sponsor in preparing comment resolutions that increase the likelihood of quick project approval at RevCom.

RevCom Conventions Confidentiality Statements and Copyright Notices on Communications 2. Changes to Balloted Drafts Prior to IEEE-SA Standards Board Approval 3. Practice regarding recommendation for Conditional Approval 4. Procedural Changes and Grandfathering for Proposed Standards 5. Recirculation Ballots in Progress at the Submittal Deadline (10 Day Practice) 6. Promises of Future Action 7. Mentored Project Submittals

Sample RevCom Comments 848/Draft 4 PE/IC 1. T W Olsen #1 -- Yes (Approve) 28-Oct-2014 11:35 A number of comments on the final ballot show disposition detail that the comment is referred to the editors. Comments r01-7 and r01-6 are not appropriate for passing to the editors as they would involve technical changes. I vote to disapprove until the comments are adequately addressed and a recirculation ballot conducted. 1.1 Patrick Gibbons (RevCom Editorial Staff) 30-Oct-2014 10:21 I have reviewed comments r01-7 and r01-6 and agree that these are not editorial in nature. 1.2 T W Olsen 05-Jan-2015 08:03 The recirculation ballot took care of this issue, and I am now satisfied. 30-Jan-2015 Agenda 575/Draft 13 PE/IC Gary Hoffman (RevCom Chair) #1 -- Yes (Approve) 12-May-2014 23:33 Disposition Status and detail is missing for MBS comment r03-2. 11-Jun-2014 Agenda

Sample RevCom Comments   386/Draft 3.2  PE/IC   1. Adrian Stephens (RevCom Member) #1 -- Yes (Approve) 16-Jun-2016 06:08 Comment i-53: according to RevCom comment resolution guidelines: "Some examples of unacceptable attachment files are: ...  Any other attachment that does not allow the CRG to address a comment as a single issue or does not related to a specific line, paragraph, figure, or equation in the balloted draft." The attachment from Mr XXX fails this test and is therefore in RevCom's view unacceptable. It would have been reasonable to reject the comment for this reason. However, the comment Disposition Detail does not state this. It states: "See the attached word document containing disposition detail for each of the commenter's proposed changes."

Sample RevCom Comments 386/Draft 3.2 PE/IC (continued) The MyBallot toolset does not support the attaching of word documents. Given that the committee wanted to substantively respond it could have done so: 1. IN the Disposition detail itself. 2. In a document put on a permanent location on a public web-server, and an URL embedded in the Disposition Detail. As it stands, I might be wrong, but I don't believe I have access to this "attachment". Please provide it to the RevCom administrator who can include it in the package of information available to RevCom reviewers. 1.1 Karen Evangelista (RevCom Administrator) 16-Jun-2016 07:32 I believe if you open the zip "1st Recirculation package - Other Ballot Document - Sent 12-Nov-2015" that it is in there. Let me know if you have trouble viewing it. 1.2 Adrian Stephens (RevCom Member) 16-Jun-2016 08:42 Thank you to Karen for pointing this out. I can now see the file. No further response is requested. 29-Jun-2016 Agenda

Questions??? Malia Zaman Program Manager m.zaman@ieee.org 732-850-6608

How to Access the Comments Download the .zip file containing a CSV file of the comments and any files that may be attached to the comments. Edit the CSV file using any spreadsheet editor and save the file in either .csv or .xls format.

Download Comments Resolution File 9/19/2018

Download Comments Resolution File (Con’td 9/19/2018

RevCom - Relevant Sites & Documents RevCom Home Page http://standards.ieee.org/about/sasb/revcom/index.html Submitting a Proposed Standard to RevCom in myProject https://mentor.ieee.org/etools_documentation/dcn/11/etools_documentation-11-0024-MYPR-revcom-submission-pres.pdf myProject webpage for standards development https://development.standards.ieee.org/ Standards Development Online http://standards.ieee.org/resources/development/final/finalmoreinfo.html IEEE-SA Working Group Chair Quick Reference Guide http://standards.ieee.org/develop/policies/ieee_sa_toolkit.pdf IEEE-SA Standards Board Operations Manual http://standards.ieee.org/develop/policies/sa_opman/index.html RevCom Conventions [Internal Committee Rules] http://standards.ieee.org/board/rev/revconventions.html RevCom Comment Resolution Preparation Guidelines http://standards.ieee.org/about/sasb/revcom/guidelines.pdf