© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. HL7 Peer Review Process Brought to you by Project Services Refer to slide notes for [footnote references] and additional info

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review- Introduction Merriam-Webster defines Peer Review as  a process by which something proposed (as for research or publication) is evaluated by a group of experts in the appropriate field [1] Peer Reviews are used in many settings, particularly academic and professional settings, with the first known Peer Review dating back to 1665 [2] Peer Review allows more ‘eyes’ on an artifact being developed

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. How Peer Reviews are used in HL7 Used for Governance and Operations Manual (GOM) maintenance (see GOM § ) Work Groups have used to mature content, resulting in ballot passage in one cycle May be used to provide input on new HL7 processes, as defined in “Introducing New Processes to HL7 ”[3]Introducing New Processes to HL7

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. HL7 Peer Review [4] The purpose of the peer review is to encourage a more complete review without going through the rigor of the ballot process The review may also cover "partial" or incomplete artifacts Peer Reviews may be scheduled outside the ballot review schedule to avoid “ballot fatigue” Peer Reviews may be used by groups [5] to solicit input and perform quality assurance

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Steps (by role) Sponsor Group 1.Prepare artifact(s) 2.Plan logistics 5.Review Comments 6.Complete Peer Review Reviewers 3.Prepare comments 4.Submit comments

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 1 - Prepare Artifact(s) An artifact is produced to a level that is deemed to be "as complete as possible without review" or ready for review outside the sponsor group, for example:  A task force group solicits input from the sponsor group  A sponsor group solicits input from other groups for specific project artifact deliverables  A group solicits input from membership, or selected “target audience”, on a new process

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 2 - Plan logistics Upon agreement from the sponsor group the author distributes the artifact, announcement, and comment instructions:  The announcement must include the review timeline and instructions to submit comments.  The author may schedule a Question & Answer (Q&A) meeting to walk through the artifact or otherwise assist reviewers; include info in the announcement.  Multiple options for the peer review comment collection are available, supporting “Simple” or “In Depth” review. [6]

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 3 - Prepare comments Reviewers go through the artifact identifying issues and concerns:  Multiple formats have been defined to collect comments, with differing degree of formality The peer review form may be similar to a ballot comment spreadsheet, but with less information collected Comments may be collected on a wiki page

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 4 - Submit comments Reviewers submit their peer review comments as directed in the announcement:  Comments must be submitted by the deadline established by the sponsor group. Tip: Sponsor Group may want to remind reviewers of the impending deadline.

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 5 - Review Comments Peer Review comments are reviewed: Focus on improving the content of the artifact being reviewed, not dispositioning comments Review approach may be “Simple” or “In Depth”  Simple - mark all items “Reviewed”; it is not necessary to categorize further  In Depth - identifying disposition for each line item (similar to ballot dispositions) Review may be by a single individual, or task group, or other designee formed by the sponsor group

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Step 6 – Complete Peer Review Simple: Apply comments to the artifact and determine publication for the next version:  Schedule the content for another peer review, possibly widening the targeted reviewers  Submit for ballot with concurrence of the sponsor group and appropriate project approvals In Depth: Schedule a call to walkthrough the comments (sponsor group and/or reviewers) Confirm resolutions, answer questions, identify rationale for rejections Disposition each line item Determine next step, e.g. subsequent peer review or ballot

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Options Peer Review forms are available at: Formats have been developed by:  Process Improvement Committee (PIC)  Less formal word document, recommended for “Simple” reviews  Project Services (PS)  Adapted from the ballot comment spreadsheet with input from the Service Aware Interoperability Framework ( SAIF) Architecture Program (AP), suitable for either “Simple” or “In Depth” reviews  Architecture Review board (ArB)  Excel spreadsheet with minimal disposition categories; recommended for “In Depth” reviews

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Form – PIC Developed by PIC

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Form – Project Services Developed by Project Services/SAIF AP This form is adapted from the Ballot Comment Spreadsheet

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Form – Project Services, cont. This form is adapted from the Ballot Comment Spreadsheet, but simplified to remove ballot specific content Simple reviews may set all dispositions to “Reviewed”

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Peer Review Form – ARB Developed by ARB

© 2011 Health Level Seven ® International. All Rights Reserved. HL7 and Health Level Seven are registered trademarks of Health Level Seven International. Reg. U.S. TM Office. Comments or Questions? Please forward comments or questions to the Project Services Co-Chairs Contact information available at:  ces/leadership.cfm ces/leadership.cfm