1. IEEE-SA Sponsor Ballot: Comment Resolution
PE/Surge Protection Devices Committee
May 24th, 2017
St Louis, MO

2. Agenda Ballot Comment Resolution What it is IEEE-SA Operations Manual
Do’s and Don’ts
Best Practices
RevCom Comments
Questions

3. Comment Resolution
After the Sponsor ballot has closed, the Working Group will need to respond to the commenters.
This can often be a daunting task for ballots that have a large number of comments.
Know the process,
what to avoid,
what to prepare, etc.

4. IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts)
Comments addressing grammar, punctuation, and style, whether attached to an affirmative or a negative vote, may be referred to the publications editor for consideration during preparation for publication.
It should be borne in mind that documents are professionally edited prior to publication.
After balloting closes, voters who have voted “Disapprove”, may “flip” their vote to either “Approve” or “Abstain” at any time before the next recirculation or the RevCom meeting.  

5. IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts)
Changes may be made in the document to resolve negative votes that are accompanied by comments or for other reasons.
All substantive changes made since the last balloted draft shall be identified and recirculated.
All unresolved negative votes with comments shall be recirculated.
During a recirculation ballot, balloting group members shall have an opportunity to change their previously cast ballots.
A change to "disapprove," which is submitted with comments, shall be based only on;
changed portions of the balloted document,
clauses affected by the changes, or
portions of the balloted document that are the subject of the unresolved negative votes.

6. IEEE-SA Ops. Manual Clause 5.4.3.2 (Excerpts)
If a change to "disapprove" is based solely on comments concerning previously approved portions of the balloted document,
the balloter shall be informed that the comments are not based on the changed portion of the balloted document and, therefore,
those comments may not be addressed in the current ballot
and may be considered for a future revision of the standard or may be considered out of scope.
If the balloter does not agree to change the negative ballot, the ballot shall be recorded with an unresolved negative without comment.
Additional recirculation ballots shall be required if negative votes with new comments within the scope of the recirculation result.
Proposed standards receiving a significant number of unresolved negative votes should be considered by the Sponsor for trial-use (see Standards Operations Manual, Section 5.7).

7. IEEE-SA Ops. Manual Clause 5.4.3.3 (Excerpts)
The Sponsor Ballot is the process whereby the Balloting group is trying to reach consensus on draft standard.
Consensus means agreement among the majority. According to IEEE rules, consensus is defined as a minimum 75% return of ballots from the balloting group, and a 75% approval rate from that 75% return group.
The Sponsor shall make a reasonable attempt to resolve all comments, objections, and negative votes that are accompanied by comments. Comments that advocate changes in the document, whether technical or editorial, may be accepted, revised, or rejected.
All comments from the Sponsor Ballot must be considered and responded to by the WG or Ballot Resolution Group (BRC), this process may change the draft which may in result in changed votes.
Negative comments typically explain any difficulties the balloter has with the current document and offer precise wording for changes that would turn Disapprove Votes to Approve Votes.
In many cases, the balloter may offer vague solutions or even no solution at all. At this point, the WG or BRC should examine the problem to see if they can resolve it on their own, or they may discuss the situation with the balloter and solicit more precise language.
If a negative vote comes without comments, it cannot be resolved.

8. Resolution Statuses Defined:
Accepted – The Ballot resolution committee (BRG) or Comment resolution group (CRG) agreed exactly with comment and change proposed by the commenter.
The proposed changes contain sufficient detail so that voters can understand the specific changes that satisfy the commenter and the editor can make the change
Response: The disposition detail field should be left blank
Revised –If the CRC agrees with the comment, but disagree with specific changes the Disposition Status should be "revised"
The CRG disagrees with all or part of the specific details in the proposed change in the comment,
the proposed change does not contain sufficient detail for the CRG to understand the changes;
changes made by the CRG contain additions or modifications to what was proposed,
Response: The disposition detail field should contain sufficient detail so that voters understand the specific changes determined by the CRG or an editor can change
9/19/2018

9. Resolution Statuses Defined:
Rejected - the CRG disagrees with the comment or the comment is out of scope.
The proposed change in the comment does not contain sufficient detail so that the CRG can understand the specific changes
the CRG cannot come to a consensus to make changes necessary to address the comment
the comment is in support of an unsatisfied previous comment associated with a disapprove vote and does not provide substantive additional rationale
the comment includes an attachment that does not meet the criteria indicated by the myBallot system;
that the CRG cannot address as a single issue; or that does not relate to a specific line, paragraph, figure, or equation in the balloted draft
or the commenter has indicated to the CRG chair that they wish to withdraw the comment
Response: The disposition detail field should explain why the comment is rejected:
an explanation of why the CRG disagrees with the comment,
a statement that the comment is out of scope, and the rationale,
a statement that the proposed change in the comment does not contain sufficient detail so that the CRG can understand the specific changes

10. Do’s and Don’ts
Don’t make, or promise to make, any post-ballot changes to the draft.  
If you do RevCom would require a recirculation.
RevCom does review the last balloted draft
Do reject an editorial comment and defer it to IEEE-Editorial for review (not for change)
You cannot make the promise that IEEE-Editorial will indeed make the change, just defer for review for change.
Example Comment: Resize Figure 1 to fit the page better
Don’t Use: Accepted. Will resize the figure.
Do Use: Rejected. Defer this change to IEEE-Editorial staff for review and change during final publication, use suggested response: “Will let Editorial staff determine if changes can possibly be made during publication.”

11. Do’s and Don’ts
For Revised or Rejected status don’t have any empty disposition status or detail columns
All comments need to be shown consideration
If “accepted” a leave the disposition detail column empty
Do have sufficient detail explaining the reason for rejection or the revised comment
Example of rejected comments:
Don’t use: Rejected. More discussion required. Where were the sensor experts when we requested assistance in writing these requirements?"
Do Use: Rejected: Commenter did not provide sufficient details or any proposed changes for the CRG to make changes to the draft.

12. Do’s and Don’ts
Don’t have an outstanding negative ballot with comment(s) and submit to RevCom without addressing the comment(s)
Do the balloter a reason for rejecting the comment and advise he/she that you are submitting to RevCom.
Do include this to RevCom in your submission

13. Do’s and Don’ts Don’t cross reference
“Disagree – see resolution of comment 1234”
Do cut and paste the comment resolution from the cited comment and then add “ (same comment resolution as comment 1234)”

14. Do’s and Don’t’s Don’t reference an external document
“Disagree – see 11-09/9876r1”
Not an appropriate response
Do copy information from document 11-09/9876r1
paste the information from the document in the disposition detail

15. Best Practices The WG Chair can form a CRG to resolve the comments
Document the formation and the scope of the Adhoc in the WG meeting minutes
Resolve the comments using the disposition statuses ; accepted, rejected, revised
Can split work and assign roles to expedite process
Ensure accepted changes are incorporated into the Draft
Note substantive changes in the cover letter (if needed)
Note substantive changes in the Draft offered for recirculation
Deleted material in the Draft should be indicated using strikethrough.
Any color may be used for the strikethrough.
Any color is acceptable to indicate changes.
Location of deleted and new material should be indicated using a change bar in the margin

16. Submission to RevCom
Final approval of a standard is achieved by submitting the document and supporting material to RevCom (Review Committee), which issues a recommendation to the IEEE-SA Standards Board
Once a project is submitted to RevCom the Working Group Chair is responsible to responding to any comments/questions raised by the RevCom Members
All Dialogue and Responses are done via myProject
WG Chairs will receive automatic notification from myProject that a comment has been made
Don’t reply to the
Do log into myProject and post a response
Don’t delay in responding to RevCom, the longer you wait, the likelihood of additional comments increases.

17. IEEE-SA RevCom Comment Resolution Preparation Guidelines
RevCom will review all the rounds of balloting, the comments, disposition status and detail along with the drafts.
RevCom will base their review, comments, and recommendations on:
RevCom Conventions
RevCom Comment Resolution Preparation Guidelines
The Guideline is intended to provide assistance to the Sponsor in preparing comment resolutions that increase the likelihood of quick project approval at RevCom.

18. RevCom Conventions
Confidentiality Statements and Copyright Notices on Communications
2. Changes to Balloted Drafts Prior to IEEE-SA Standards Board Approval
3. Practice regarding recommendation for Conditional Approval
4. Procedural Changes and Grandfathering for Proposed Standards
5. Recirculation Ballots in Progress at the Submittal Deadline (10 Day Practice)
6. Promises of Future Action
7. Mentored Project Submittals

19. Sample RevCom Comments
848/Draft 4 PE/IC 1. T W Olsen #1 -- Yes (Approve) 28-Oct :35 A number of comments on the final ballot show disposition detail that the comment is referred to the editors. Comments r01-7 and r01-6 are not appropriate for passing to the editors as they would involve technical changes. I vote to disapprove until the comments are adequately addressed and a recirculation ballot conducted. 1.1 Patrick Gibbons (RevCom Editorial Staff) 30-Oct :21 I have reviewed comments r01-7 and r01-6 and agree that these are not editorial in nature. 1.2 T W Olsen 05-Jan :03 The recirculation ballot took care of this issue, and I am now satisfied. 30-Jan-2015 Agenda 575/Draft 13 PE/IC Gary Hoffman (RevCom Chair) #1 -- Yes (Approve) 12-May :33 Disposition Status and detail is missing for MBS comment r Jun-2014 Agenda

20. Sample RevCom Comments
  386/Draft 3.2  PE/IC   1. Adrian Stephens (RevCom Member) #1 -- Yes (Approve) 16-Jun :08 Comment i-53: according to RevCom comment resolution guidelines: "Some examples of unacceptable attachment files are: ...  Any other attachment that does not allow the CRG to address a comment as a single issue or does not related to a specific line, paragraph, figure, or equation in the balloted draft." The attachment from Mr XXX fails this test and is therefore in RevCom's view unacceptable. It would have been reasonable to reject the comment for this reason. However, the comment Disposition Detail does not state this. It states: "See the attached word document containing disposition detail for each of the commenter's proposed changes."

21. Sample RevCom Comments
386/Draft 3.2 PE/IC (continued) The MyBallot toolset does not support the attaching of word documents. Given that the committee wanted to substantively respond it could have done so: 1. IN the Disposition detail itself. 2. In a document put on a permanent location on a public web-server, and an URL embedded in the Disposition Detail. As it stands, I might be wrong, but I don't believe I have access to this "attachment". Please provide it to the RevCom administrator who can include it in the package of information available to RevCom reviewers. 1.1 Karen Evangelista (RevCom Administrator) 16-Jun :32 I believe if you open the zip "1st Recirculation package - Other Ballot Document - Sent 12-Nov-2015" that it is in there. Let me know if you have trouble viewing it. 1.2 Adrian Stephens (RevCom Member) 16-Jun :42 Thank you to Karen for pointing this out. I can now see the file. No further response is requested. 29-Jun-2016 Agenda

22. Questions???
Malia Zaman Program Manager

23. How to Access the Comments
Download the .zip file containing a CSV file of the comments and any files that may be attached to the comments.
Edit the CSV file using any spreadsheet editor and save the file in either .csv or .xls format.

24. Download Comments Resolution File
9/19/2018

25. Download Comments Resolution File (Con’td
9/19/2018

26. RevCom - Relevant Sites & Documents
RevCom Home Page
Submitting a Proposed Standard to RevCom in myProject
myProject webpage for standards development
Standards Development Online
IEEE-SA Working Group Chair Quick Reference Guide
IEEE-SA Standards Board Operations Manual
RevCom Conventions [Internal Committee Rules]
RevCom Comment Resolution Preparation Guidelines