1. Development of the Assessor Governance Framework at the College of Physicians and Surgeons of Ontario (Canada)

2. Declaration of Conflict of Interest Faculty: Nanci Harris Relationships with commercial interests: None

3. Presentation Overview Background Network Development Governance Framework Questions

4. Context Since 1980, medical regulation in Ontario has been supported and strengthened by physician assessment. Beginning in 2010, CPSO physician assessors were organized into specialty networks and a framework was developed to ensure consistent, high quality assessor recruitment, training and evaluation.

5. CPSO Assessment Programs The CPSO has a number of assessment programs for which we use assessors, including: Peer Assessments (which include peer re-assessments, and comprehensive re-assessments in the form of PREP ™ or Specialties Assessment) IHF Assessments Methadone Assessments Registration Assessments Assessments for Change in Scope of Practice and Re-Entry to Practice Out-Of-Hospital Premises Inspection Program (OHPIP) Investigations and Resolutions

6. Network Development In 2010, CPSO created a new staff position to support 550+ physicians who conduct 2600 assessments annually. This position led to development of an Assessor Governance Framework that describes the values and criteria by which assessors are recruited, trained and evaluated. New assessor training includes e-Learning modules, 1:1 instruction, newsletters, webinars, ongoing feedback, and engagement in evaluation and process improvement activities. To evaluate the new model, ongoing feedback has been collected from staff, assessors and committee-members to identify opportunities for improvement.

7. Governance Framework The role of the assessor is integral in meeting the legislative requirements to practice medicine in Ontario. Providing assessors support in a framework that offers training, resources and opportunities for networking will facilitate continued proficiency and will help to promote commitment to medical regulation. This is consistent with the stated Values of the Profession and the Canadian Medical Association’s Code of Ethics.Values of the Profession Canadian Medical Association’s Code of Ethics

8. Framework Components A.Recruitment Criteria B.Orientation C.Ongoing Training & Engagement D.Evaluation E.Conduct/Behaviour F.Conflict of Interest/Apprehension of Bias G.Resources H.Delisting and Retirement/Resignation

9. Current Status Since 2010, 325 assessors have been trained for a total of 650 assessors. An analysis of stakeholder feedback indicated: assessors better understand CPSO processes and feel more confident; improved consistency in assessor reports; increased positive feedback from assessed physicians; and more engaged assessors. Identified opportunities for improvement include increasing assessor engagement and training.

10. Conclusions The assessor network and governance framework have collectively improved assessor performance. Even long-term assessors (recruited pre-2010) evaluated the new training as useful. Future initiatives include expanding training and engagement opportunities for assessors, including how to better provide practice-improvement feedback to assessed physicians. Standardized training and engagement of physician assessors has improved assessor performance. Building on these findings, the CPSO plans to increase the breadth of training with a focus on facilitating feedback around practice-improvement.

12. College of Physicians and Surgeons of Ontario Orientation Modules – All Assessors Assessor Orientation modules The three Assessor Orientation modules include the following: Module 1: Why the Assessor Role is Important to the CPSO Module 2: How the Assessor Supports the Work of the College Module 3: How the College Supports the Assessor in the Role Now also part of new assessor training

13. College of Physicians and Surgeons of Ontario Orientation & Training for Peer Assessors Orientation that is general to all assessors: Brief overview of College assessment types Conflicts of interest and/or apprehension of bias Confidentiality, privacy and breaches Assessor report storage and record retention Importance of language/attitude CPSO policies and other reference resources Continuing Professional Development (CPD)

14. College of Physicians and Surgeons of Ontario Network Groups and Assessors Did you know? Across the Quality Management Division there are: Over 650 assessors (including physicians, nurses and technologists) Placed into 52 network groups Within peer assessment, there are: Over 450 peer assessors (all physicians) Placed into 33 network groups

15. College of Physicians and Surgeons of Ontario The Assessor Network Mandate: Provide feedback to continuously improve College assessments Foster a community of College assessors Provide advice and guidance to the College Community of Assessors