University of Manitoba Pre-Survey Meeting with Resident Representatives & Senior Residents Date: July 3, 2013 Time: 2:00 to 4:00 p.m. Room: Pharmacy Apotex, Theatre # 264

Objectives of the Meeting To review the: Accreditation Process Categories of Accreditation Standards of Accreditation Role of residents in the accreditation process

Program Administration Member of the Residency Program Committee Must be elected Communication to and from Residency Program Committee Evaluations Evaluation of the program Rotations, teachers, teaching Understand the Standards Feedback on how program is meeting standards Role of Residents in the Accreditation Process

Continuing quality improvement process Peer-reviewed Medical education experts Outside discipline Based on Standards Includes competency framework Categories of Accreditation Regular Schedule Principles of Accreditation

Internal Reviews Monitoring Six Year Survey Cycle

Chair - Dr. Sarkis Meterissian –Responsible for general conduct of survey Deputy chair – Dr. Maureen Topps –Visits teaching sites / hospitals Surveyors Resident representatives – CAIR Regulatory authorities representative – FMRAC Teaching hospital representative – ACAHO The Survey Team

Questionnaires and appendices –Completed by program Program-specific Standards (OTR/STR/SSA) Report of last regular survey Specialty Committee comments –Also sent to PGD / PD prior to visit Exam results for last six years Reports of mandated Royal College reviews since last regular survey, if applicable Information Given to Surveyors

Includes: Document review (30 min) Meetings with: –Program director (75 min) –Department head (30 min) –Residents – per group of 20 (60 min) –Teaching staff (60 min) –Residency Program Committee (60 min) The Survey Schedule

Document review (30 min) Residency Program Committee Minutes Resident Assessments Files The Survey Schedule

Program director Overall view of program Evaluation of Standards Department head Support for program Resources available to program Teaching faculty Involvement with residents Communication with program director Meeting Overview

Group(s) of 20 residents (60 min) If off-site, tele- or video- conferencing Looking for balance of strengths & challenges Focus on Standards Evaluate the learning environment Meeting with ALL Residents

–Objectives –Educational experiences –Service /education balance –Increasing professional responsibility –Academic program / protected time –Supervision –Assessments of resident performance –Evaluation of program / assessment of faculty –Career counseling –Educational environment –Safety Topics to discuss with residents

Complete the CAIR questionnaire Confidential, not given to survey team Meet together as a group to discuss the strengths & challenges of your program 1 to 2 months before survey Obtain a copy of the pre-survey questionnaires and the previous survey report If you feel you need more time with surveyor, request it Be open and honest with surveyor Comments in meetings are anonymous Preparing for the Survey Role of the Resident

All members of RPC attend meeting, including resident members Review Committee responsibilities Opportunity for surveyor to provide feedback on information obtained during survey Meeting with Residency Program Committee

Survey team discussion –Evening following review Feedback to program director –Exit meeting with surveyor –Survey team recommendation Category of accreditation Strengths & weaknesses The Recommendation

New terminology – June 2012 Approved by the Royal College, CFPC and CMQ. Categories of Accreditation

Accredited program Follow-up: –Next regular survey –Progress report (Accreditation Committee) –Internal review –External review Accredited program on notice of intent to withdraw accreditation Follow-up: –External review Categories of Accreditation

Accredited program with follow-up at next regular survey –Program demonstrates acceptable compliance with standards. Categories of Accreditation Definitions

Accredited program with follow-up by College-mandated internal review –Major issues identified in more than one Standard –Internal review of program required and conducted by University –Internal review due within 24 months Categories of Accreditation Definitions

Accredited program with follow-up by external review –Major issues identified in more than one Standard AND concerns - are specialty-specific and best evaluated by a reviewer from the discipline, OR have been persistent, OR are strongly influenced by non-educational issues and can best be evaluated by a reviewer from outside the University –External review conducted within 24 months –College appoints a 2-3 member review team –Same format as regular survey Categories of Accreditation Definitions

Accredited program on notice of intent to withdraw accreditation –Major and/or continuing non-compliance with one or more Standards which calls into question the educational environment and/or integrity of the program –External review conducted by 3 people (2 specialists + 1 resident) within 24 months –At the time of the review, the program will be required to show why accreditation should not be withdrawn. Categories of Accreditation Definitions

SURVEY TEAM ROYAL COLLEGE SPECIALTY COMMITTEE ACCREDITATION COMMITTEE Reports Reports & Responses Recommendation Reports Responses After the Survey Report & Response UNIVERSITY

Chair + 16 members Ex-officio voting members (6) –Collège des médecins du Québec (1) –Medical Schools (2) –Resident Associations (2) –Regulatory Authorities (1) Observers (9) –Collège des médecins du Québec (1) –Resident Associations [CAIR & FMRQ] (2) –College of Family Physicians of Canada (1) –Regulatory Authorities (1) –Teaching Hospitals (1) –Resident Matching Service (1) –Accreditation Council for Graduate Medical Education (2) The Accreditation Committee

All pre-survey documentation available to the surveyor Survey report Program response Specialty Committee recommendation History of the program Information Available to the Accreditation Committee

Decisions –Accreditation Committee meeting May/June 2014 Dean & postgraduate dean attend –Sent to University Specialty Committee Appeal process is available The Accreditation Committee

“A” Standards Apply to University, specifically the PGME office “B” Standards Apply to EACH residency program “C” Standards Apply to Areas of Focused Competence (AFC) programs General Standards of Accreditation

A1University Structure A2Sites for Postgraduate Medical Education A3Liaison between University and Participating Institutions “A” Standards

B1Administrative Structure B2Goals & Objectives B3Structure and Organization of the Program B4Resources B5Clinical, Academic & Scholarly Content of the Program B6Assessment of Resident Performance “B” Standards

There must be an appropriate administrative structure for each residency program. Program director Time & support Residency Program Committee Representative from each site and major component Resident member(s) - Must include at least ONE elected resident Meets regularly, four times a year - Minutes B1 – Administrative Structure

Responsibilities of the Residency Program Committee Selection, assessment & promotion of residents Ongoing review of program Assessment of program / teachers / rotations Research environment Appeal mechanism Career & stress counseling Resident safety B1 – Administrative Structure

Program director autocratic Residency Program Committee dysfunctional –Unclear Terms of Reference (membership, tasks and responsibilities) Agenda and minutes poorly structured Poor attendance –Department head unduly influential –RPC is conducted as part of a Dept/Div meeting No resident voice B1 – Administrative Structure “Pitfalls”

B2 – Goals & Objectives There must be a clearly worded statement outlining the goals of the residency program and the educational objectives of the residents. Rotation-specific Structured to reflect CanMEDS Competencies Circulated to residents & teaching staff Used in planning and assessment of residents

CanMEDS Competencies Medical Expert Communicator Collaborator Manager Health Advocate Scholar Professional

B2 – Goals & Objectives “Pitfalls” Missing CanMEDS roles in overall structure –Okay to have rotations in which all CanMEDS roles may not apply (research, certain electives) Goals and objectives not used by faculty/residents Goals and objectives dysfunctional – does not inform assessment Goals and objectives not reviewed regularly

There must be an organized program of rotations and other educational experiences, both mandatory and elective, designed to fulfill the educational requirements and allow residents to achieve competence in the specialty. Include all components of specialty Equivalent opportunity Senior residency B3 – Structure & Organization of the Program

Increasing professional responsibility Appropriate supervision Balance of service and education Safe learning/educational environment Promotes resident safety Free from intimidation, harassment or abuse B3 – Structure & Organization of the Program

Graded responsibility absent Service/education imbalance –Service provision by residents should have a defined educational component including evaluation Educational environment poor B3 – Structure & Organization “Pitfalls”

There must be sufficient resources to provide the opportunity for all residents to achieve the educational objectives. Teaching faculty Variety & number of patients, specimens and procedures Physical and technical facilities Inpatient, ambulatory, emergency, ICU Educational B4 – Resources

Insufficient faculty for teaching/ supervision Insufficient clinical/technical resources Infrastructure inadequate B4 – Resources “Pitfalls”

There must be a clinical, academic and scholarly program that prepares residents to fulfill all the roles of the specialist. Academic program Organized curriculum Organized teaching in basic & clinical sciences Evidence of teaching the CanMEDS Competencies Attendance Staff, residents B5 – Clinical, Academic & Scholarly Content of Program

Organized academic curriculum lacking or entirely resident driven –Poor attendance by residents and faculty Teaching of essential CanMEDS roles missing Role modelling is the only teaching modality B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls”

There must be mechanisms to ensure systematic assessment of each resident. Based on goals & objectives Uses appropriate and varied assessment methods Feedback Formal, timely, appropriate Face-to-face Adequately documented B6 – Assessment of Resident Performance

Mechanism to monitor, promote, remediate residents lacking Formative feedback not provided and/or documented Assessments not timely, not face to face Summative assessment (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges B6 – Assessment of Resident Performance “Pitfalls”

University of Manitoba On-site Survey February 23 to 28,

Office of Education Sarah Taber Assistant Director Education Strategy & Accreditation Educational Standards Unit Sylvie Lavoie Survey Coordinator Contact Information at the Royal College