1. Resident Representatives
University of Alberta
Pre-survey Visit
March 16, 2017

2. Objectives of the Meeting
To review the:
Accreditation Process
Role of residents in the accreditation process
Standards of Accreditation
Categories of Accreditation

3. Accreditation Is a process to:
Improve the quality of postgraduate medical education
Provide a means of objective evaluation of residency programs for the purpose of Royal College accreditation
Assist program directors in reviewing their own program

4. Principles of Accreditation
Based on General and Specific Standards
Continuing quality improvement process
Peer-reviewed
Medical education experts
Outside discipline
Includes competency framework
Categories of Accreditation 4

5. Role of Residents: The Accreditation Process
Program Administration
Member of the Residency Program Committee
Must be elected
Communication to and from Residency Program Committee
Evaluations
Evaluation of the program
Rotations, teachers, teaching
Understand the standards
Feedback on how program is meeting standards

6. Role of Residents: Preparing for the Survey
Complete the RDoC pre-accreditation questionnaire
Confidential, not given to survey team
Meet together as a group to discuss the strengths and challenges of your program
One to two months before survey
If you feel you need more time with surveyor, request it
Be open and honest with surveyor
Comments in meetings are anonymous
Presented by RDoC representative 7

7. RDoC Pre-Accreditation Questionnaire
As part of the accreditation process, the Professional Association of Resident Physicians of Alberta (PARA) sends an anonymous, confidential RDoC pre- accreditation questionnaire to all residents in programs prior to the onsite survey.
The RDoC questionnaire will be sent to graduating residents before July. The questionnaire will be sent to rest of residents by September.
Results are collated and a report written by PARA and then sent to RDoC to review.
Final report is given ONLY to the resident surveyors on the Royal College and CFPC survey teams.
Presented by RDoC representative

8. RDoC Pre-Accreditation Questionnaire
Used since the early 1980s to ensure that residents’ perspectives and concerns are accurately and adequately voiced, in a confidential manner.
This questionnaire is specifically designed to obtain residents’ perspectives on their training.
The resident surveyors can convey, in conversation with the chair of the survey team, the anonymous, collated themes which will help the team during the onsite visit.
RDoC is assessing the value of providing to the survey team, a brief, high-level, de-identified summary report of survey themes for the onsite visit.
Presented by RDoC representative 9

9. RDoC Pre-Accreditation Questionnaire
The ongoing use of the RDoC Pre- Accreditation Survey ensures:
Resident accreditation data is primarily collected externally and kept arms-length from Faculties and certifying bodies;
Resident data is confidential and anonymized; and
Residents have the opportunity to provide qualitative as well as quantitative feedback as part of the accreditation process.
Presented by RDoC representative

10. RDoC Pre-Accreditation Questionnaire
Residents are critical to the process, and our input is highly valued
Accreditation provides residents the mechanism to effect positive changes on our residency programs that in many situations will result in significant long-term benefits
The RDoC pre-accreditation questionnaire is only one tool that is available to the resident surveyors to help them understand the resident on-the-ground perspective on his/her training
Presented by RDoC representative 11

11. University of Alberta Onsite Survey
November 26 to December 1, 2017 12

12. The Survey Team Chair – Dr. Glen Bandiera
Responsible for general conduct of survey
Deputy chair – Dr. Alan Chaput
Visits teaching sites / hospitals
Surveyors
Program reviews
Resident representatives – RDoC
Regulatory authorities representative – FMRAC
Teaching hospital representative – HealthCareCAN 13

13. Information Given to Surveyors
Questionnaires and appendices
Completed by program
Program-specific standards (OTR/STR/SSA)
Report of last regular survey
Specialty Committee comments
Also sent to PGD / PD prior to visit
Exam results for last six years
Reports of mandated Royal College reviews since last regular survey, if applicable 14

14. The Survey Schedule Includes: Document review (45 min) Meetings with:
Program director (75 min)
Department head (30 min)
Residents – per group of up to 20 (60 min)
Teaching staff (60 min)
Residency Program Committee (60 min) 15

15. Meeting Overview Document review (45 min)
Residency Program Committee minutes
Resident assessments files 16

16. Meeting Overview Program director Department head Teaching faculty
Overall view of program
Evaluation of standards
Department head
Support for program
Resources available to program
Teaching faculty
Involvement with residents
Communication with program director 17

17. Meeting with Residents
Accreditation is the opportunity for residents in the programs to openly and honestly evaluate, and help improve the quality of your residency program
Feedback given to the survey team will help your program continue to promote areas of strength and at the same time, focus on areas that need improvement
Key questions to keep in mind:
What are the program strengths?
What are the areas that need improvement?
What can accreditation do to improve your residency program?
What resources do you need?
Presented by RDoC representative 18

18. Meeting with Residents
Ask for strengths and weaknesses of program
Do you understand the objectives of program?
Do you have a voice, feel part of program?
What is your involvement with the academic program?
Presented by RDoC representative 19

19. Meeting with Residents
Communication links to program
Program director – accessibility
RPC – are they informed?
RPC – is resident feedback taken into account and subsequent changes made?
Assessments
Teachers - have they received their assessments?
Residents
Face-to-face
Is teaching valued?
Safety and service/education balance
Identify Positives/Challenges
If you need more time with the surveyor, ask for it
To repeat: Comments in meetings are anonymous
Presented by RDoC representative 20

20. Meeting with Residency Program Committee
All members of RPC attend meeting, including resident members
Review committee responsibilities
Opportunity for surveyor to provide feedback on information obtained during survey 21

21. The Recommendation Survey team discussion Feedback to program director
Following review via survey team meeting
Feedback to program director
Exit meeting with surveyor
Survey team recommendation
Category of accreditation
Strengths and weaknesses 22

22. Categories of Accreditation
Approved by the Royal College, CFPC and CMQ
Accredited program
Follow-up:
Next regular survey
Progress report (* a category determined only by the Residency Accreditation Committee)
Internal review
External review
Accredited program on notice of intent to withdraw accreditation 23

23. Categories of Accreditation Definitions
Accredited program with follow-up at next regular survey
Program demonstrates acceptable compliance with standards

24. Categories of Accreditation Definitions
Accredited program with follow-up by College- mandated internal review
Major issues identified in more than one standard
Internal review of program required and conducted by University
Internal review due within 24 months 25

25. Categories of Accreditation Definitions
Accredited program with follow-up by external review
Major issues identified in more than one standard AND concerns -
are specialty-specific and best evaluated by a reviewer from the discipline, OR
have been persistent, OR
are strongly influenced by non-educational issues and can best be evaluated by a reviewer from outside the University
External review conducted within 24 months
College appoints a two to three member review team
Same format as regular survey; program-specific 26

26. Categories of Accreditation Definitions
Accredited program on notice of intent to withdraw accreditation
Major and/or continuing non-compliance with one or more standards which calls into question the educational environment and/or integrity of the program
External review conducted by three people
(two specialists + one resident) within 24 months
At the time of the review, the program will be required to show why accreditation should not be withdrawn. 27

27. RESIDENCY accreditation committee
After the Survey
survey team
Reports
specialty committee
royal college
university
Report &
Response
Reports
Responses
Recommendation
Reports & Responses
RESIDENCY accreditation committee 28

28. Residency Accreditation Committee
Voting members (24):
Chair and Vice Chair of the Residency AC
Chair of the International residency Program Review (1)
Association of Faculties of Medicine of Canada (2)
Collège des médecins du Québec (1)
Resident Associations (2)
Regulatory Authorities (1)
Fellows (17 including Chair and Vice Chair)
Non-voting / observers (10)
HealthCareCAN
Accreditation Council for Graduate Medical Education
Association of Faculties of Medicine of Canada
Canadian Resident Matching Service
College of Family Physicians of Canada
Collège des médecins du Québec
Committee on the Accreditation of Canadian Medical Schools
Resident Associations

29. Information Available to the Residency Accreditation Committee
All pre-survey documentation available to the surveyor
Survey report
Program response
Specialty Committee recommendation
History of the program 30

30. Residency Accreditation Committee
Decisions
Residency Accreditation Committee meeting
June 2018
Dean and postgraduate dean attend
Sent to
University
Specialty Committee
Appeal process is available (within 60 days) 31

31. General Standards of Accreditation
“A” Standards
Apply to University, specifically the PGME office
“B” Standards
Apply to EACH residency program
“C” Standards
Apply to Areas of Focused Competence (AFC) diploma programs 32

32. “A” Standards
A1 University Structure A2 Sites for Postgraduate Medical Education A3 Liaison between University and Participating Institutions 33

33. “B” Standards
B1 Administrative Structure B2 Goals and Objectives B3 Structure and Organization of the Program B4 Resources B5 Clinical, Academic and Scholarly Content of the Program B6 Assessment of Resident Performance 34

34. B1 – Administrative Structure
There must be an appropriate administrative structure for each residency program.
Program director
Time and support
Residency Program Committee
Representative from each site and major component
Resident member(s)
Must include at least ONE elected resident
Meets regularly, four times a year
Minutes 35

35. B1 – Administrative Structure
Responsibilities of the Residency Program Committee
Selection, assessment and promotion of residents
Ongoing review of program
Assessment of program / teachers / rotations
Research environment
Appeal mechanism
Career and stress counseling
Resident safety 36

36. B1 – Administrative Structure “Potential Pitfalls”
Residency Program Committee dysfunctional
Unclear Terms of Reference (membership, tasks and responsibilities)
Agenda and minutes poorly structured
Poor attendance
Department head unduly influential
RPC is conducted as part of a Dept/Div meeting
No resident voice 37

37. B2 – Goals and Objectives
There must be a clearly worded statement outlining the goals of the residency program and the educational objectives of the residents.
Rotation-specific
Structured to reflect CanMEDS Competencies
Circulated to residents and teaching staff
Used in planning and assessment of residents 38

38. CanMEDS Competencies Medical Expert Communicator Collaborator Leader
Health Advocate
Scholar
Professional 39

39. B2 – Goals and Objectives “Potential Pitfalls”
Missing CanMEDS Roles in overall structure
Okay to have rotations in which all CanMEDS Roles may not apply (research, certain electives)
Goals and objectives not used by faculty/residents
Goals and objectives dysfunctional – does not inform assessment
Goals and objectives not reviewed regularly 40

40. B3 – Structure and Organization of the Program
There must be an organized program of rotations and other educational experiences, both mandatory and elective, designed to fulfill the educational requirements and allow residents to achieve competence in the specialty.
Include all components of specialty
Equivalent opportunity
Senior residency 41

41. B3 – Structure and Organization of the Program
Increasing professional responsibility
Appropriate supervision
Balance of service and education
Safe learning/educational environment
Promotes resident safety
Free from intimidation, harassment or abuse 42

42. B3 – Structure and Organization “Potential Pitfalls”
Graded responsibility absent
Service/education imbalance
Service provision by residents should have a defined educational component including evaluation
Educational environment poor 43

43. B4 – Resources Teaching faculty
There must be sufficient resources to provide the opportunity for all residents to achieve the educational objectives.
Teaching faculty
Variety and number of patients, specimens and procedures
Physical and technical facilities
Inpatient, ambulatory, emergency, ICU
Educational 44

44. B4 – Resources “Potential Pitfalls”
Insufficient faculty for teaching/ supervision
Insufficient clinical/technical resources
Infrastructure inadequate 45

45. B5 – Clinical, Academic and Scholarly Content of Program
There must be a clinical, academic and scholarly program that prepares residents to fulfill all the roles of the specialist.
Academic program
Organized curriculum
Organized teaching in basic and clinical sciences
Evidence of teaching the CanMEDS Competencies
Attendance
Staff, residents 46

46. B5 – Clinical, Academic and Scholarly Content of Program “Potential Pitfalls”
Organized academic curriculum lacking or entirely resident driven
Poor attendance by residents and faculty
Teaching of essential CanMEDS Roles missing
Role modeling is the only teaching modality 47

47. B6 – Assessment of Resident Performance
There must be mechanisms to ensure systematic assessment of each resident.
Based on goals and objectives
Uses appropriate and varied assessment methods
Feedback
Formal, timely, appropriate
Face-to-face
Adequately documented 48

48. B6 – Assessment of Resident Performance “Potential Pitfalls”
Mechanism to monitor, promote, remediate residents lacking
Formative feedback not provided and/or documented
Assessments not timely, not face-to-face
Summative assessment (ITER) inconsistent with formative feedback, unclearly documents concerns/challenges 49

49. Conjoint Residency Education Accreditation System Reform

50. The Canadian Residency Accreditation Consortium (CanRAC)
Collaborative initiative between the Royal College, CFPC, and CMQ
New system of residency accreditation:
21st century best practices in accreditation;
Digitized; and,
Aligned with the shift towards competency-based medical education (CBME). 51
11/14/2018

51. 52

52. Shadow Surveyors Objective Role
To leverage the knowledge and expertise of experienced surveyors acting as “shadow surveyors” at an onsite survey and collect the necessary feedback on the new standards and process in accordance to the accreditation reform objectives
Role
Evaluate and validate new standards and process during an onsite survey visit
An “overlay” to the visit for selected programs or the institution to test the process with no impact to accreditation status / recommendations or decisions 53

53. Shadow Surveyor Code of Conduct
The shadow surveyor:
will act strictly as an observer during the onsite visit
will not engage in the discussions at various meetings, will not ask questions or seek clarification
will not influence in any way the recommendation of the surveyor and the survey team during the evening discussions
will not have voting privileges on the program‘s accreditation status recommendation 54

54. University of Alberta Onsite Survey
November 26 to December 1, 2017