1. Department Chairs and Division Heads
University of Alberta
Pre-survey Visit
March 16, 2017

2. Objectives of the Meeting
To review the:
Accreditation Process
Categories of Accreditation
Standards of Accreditation
Role of the:
Program directors
Department heads
Residents
Program administrators

3. Accreditation Is a process to: Based on standards
Improve the quality of postgraduate medical education
Provide a means of objective evaluation of residency programs for the purpose of Royal College accreditation
Assist program directors in reviewing their own program
Based on standards

4. Principles of the Accreditation Process
Based on General and Specific Standards
Based on Competency Framework
Onsite regular surveys
Peer-review
Input from specialists
Categories of Accreditation

5. Pre-Survey Process University Royal College Specialty Committee
Questionnaires
Specialty Committee
Questionnaires
Royal College
Comments
Questionnaires and
Comments
Comments
Program Director
Surveyor

6. Role of the Specialty Committee
Prescribe requirements for specialty education
Program standards
Objectives of training
Specialty training requirements
Examination processes
FITER
Evaluates program resources, structure and content for each accreditation review
Recommends a category of accreditation to the Accreditation Committee

7. Composition of a Specialty Committee
Voting Members (chair + 5)
Canada-wide representation
Non-voting Members
Chairs of exam committee
National Specialty Society (NSS)
ALL program directors

8. The Onsite Survey Team Chair – Dr. Glen Bandiera
Responsible for general conduct of survey
Deputy Chair – Dr. Alan Chaput
Visits teaching sites / hospitals
Surveyors
Resident representatives (RDoC)
Regulatory authorities representative (FMRAC)
Teaching hospital representative (HealthCareCAN)
Other observers
DENIS

9. Resident Survey
As part of the accreditation process the resident associations send an anonymous, confidential survey to all residents in all programs four to six months prior to the onsite survey
Results collated, reports are written by the provincial organization president or designate which are sent to RDoC/FMRQ president to review
Final Royal College accreditation report given ONLY to resident representatives on the survey team

10. Role of the Surveyor
Evaluate how the program is meeting standards at the time of survey
Looking for ‘evidence’

11. Information Given to Surveyors
Questionnaires and appendices
Completed by program
Program-specific standards (OTR/STR/SSA)
Report of last regular survey
Plus report of mandated Royal College review since last regular survey, if applicable
Specialty Committee comments
Also sent to PGD / PD prior to visit
Exam results for last six years

12. The Survey Schedule Includes: Document review (45 min) Meetings with:
Program director (75 min)
Department head (30 min)
Residents - per groups of 20 (60 min)
Teaching faculty (60 min)
Residency Program Committee (60 min)
DENIS
There is some flexibility – the meetings highlighted in grey can be moved around but other meetings needs to follow given sequence

13. Meeting Overview Program director Department heads Teaching faculty
Overall view of program, strengths, challenges and weaknesses
Address each standard
Resources to support program director and program
Department heads
Support for program director and program
Concerns regarding program
Resources available to program
Research environment
Teaching faculty
Involvement with residents
Communication with program director

14. Meeting with ALL Residents
Topics to discuss with residents
Objectives
Educational experiences
Service / education balance
Increasing professional responsibility
Academic program / protected time
Supervision
Assessments of resident performance
Evaluation of program / assessment of faculty
Career counseling
Educational environment
Safety

15. Meeting with Residency Program Committee
Program director attends first half of meeting
All members of RPC attend, including resident members
Review of responsibilities of committee
Opportunity for surveyor to provide feedback on information obtained during previous meetings

16. The Recommendation Survey team discussion Feedback to program director
Typically evening following review
Feedback to program director
Exit meeting with surveyor
Typically the morning after review
Survey team recommendation
Category of accreditation
Strengths and weaknesses

17. Categories of Accreditation
Approved by the Royal College, CFPC and CMQ
Accredited program
Follow-up:
Next regular survey
Progress report (* a category determined only by the Accreditation Committee)
Internal review
External review
Accredited program on notice of intent to withdraw accreditation

18. Categories of Accreditation Definitions
Accredited program with follow-up at next regular survey
Program demonstrates acceptable compliance with standards

19. Categories of Accreditation Definitions
Accredited program with follow-up by College- mandated internal review
Major issues identified in more than one standard
Internal review of program required and conducted by University
Internal review report due within 24 months

20. Categories of Accreditation Definitions
Accredited program with follow-up by external review
Major issues identified in more than one standard AND concerns -
are specialty-specific and best evaluated by a reviewer from the discipline, OR
have been persistent, OR
are strongly influenced by non-educational issues and can best be evaluated by a reviewer from outside the University
External review conducted by two to three people within 24 months
Same format as regular survey

21. Categories of Accreditation Definitions
Accredited program on notice of intent to withdraw accreditation
Major and/or continuing non-compliance with one or more standards which calls into question the educational environment and/or integrity of the program
External review conducted by three people
(two specialists + one resident) within 24 months
At the time of the review, the program will be required to show why accreditation should not be withdrawn

22. RESIDENCY accreditation committee
After the Survey
survey team
Reports
specialty committee
royal college
university
Report and
Response
Reports
Responses
Recommendation
Reports and Responses
RESIDENCY accreditation committee

23. Residency Accreditation Committee
Voting members (24):
Chair and Vice Chair of the Residency AC
Chair of the International residency Program Review (1)
Association of Faculties of Medicine of Canada (2)
Collège des médecins du Québec (1)
Resident Associations (2)
Regulatory Authorities (1)
Fellows (17 including Chair and Vice Chair)
Non-voting / observers (10):
HealthCareCAN
Accreditation Council for Graduate Medical Education
Association of Faculties of Medicine of Canada
Canadian Resident Matching Service
College of Family Physicians of Canada
Collège des médecins du Québec
Committee on the Accreditation of Canadian Medical Schools
Resident Associations

24. Information Available to the Residency Accreditation Committee
All pre-survey documentation available to the surveyor
Survey report
Program response
Specialty Committee recommendation
History of the program

25. Residency Accreditation Committee
Decisions
Residency Accreditation Committee meeting
June 2018
Dean and postgraduate dean attend
Sent to
University
Specialty Committee
Appeal process is available (within 60 days)

26. Questions?

27. General Standards of Accreditation
“A” Standards
Apply to University, specifically the PGME office
“B” Standards
Apply to EACH residency program
“C” Standards
Apply to Areas of Focused (AFC) diploma programs

28. “A” Standards
A1 University Structure A2 Sites for Postgraduate Medical Education A3 Liaison between University and Participating Institutions
DENIS

29. “B” Standards
B1 Administrative Structure B2 Goals and Objectives B3 Structure and Organization of Program B4 Resources B5 Clinical, Academic and Scholarly Content of Program B6 Assessment of Resident Performance
DENIS

30. “C” Standards
C1 Administrative Structure C2 Resources C3 Educational Program C4 Competency-based Assessment of Trainee Performance
DENIS

31. B1 – Administrative Structure
There must be an appropriate administrative structure
for each residency program.
Program director
Time and support
Acceptable qualifications
Residency Program Committee
Operation of the program
Program and resident evaluations
Appeal process
Selection and promotions of residents
Process for teaching and assessment of competencies
Research
Regular review of program
Faculty assessments

32. B1 – Administrative Structure “Pitfalls”
Residency Program Committee dysfunctional
Unclear Terms of Reference (membership, tasks and responsibilities)
Agenda and minutes poorly structured
Poor attendance
Department head unduly influential
RPC is conducted as part of a Dept/Div meeting
No resident voice

33. B2 – Goals and Objectives
There must be a clearly worded statement outlining the Goals and Objectives of the residency program.
Rotation-specific
Address all CanMEDS Roles
Functional / used in:
Planning
Resident assessment
Distributed to residents and faculty
Reviewed regularly
At least every 2 years

34. B2 – Goals and Objectives “Pitfalls”
Missing CanMEDS Roles in overall structure
Okay to have rotations in which all CanMEDS Roles may not apply (research, certain electives)
Goals and objectives not used by faculty/residents
Goals and objectives dysfunctional – does not inform assessment
Goals and objectives not reviewed regularly

35. B3 – Structure and Organization
There must be an organized program of rotations and other educational experiences to cover the educational requirements of the specialty.
Increasing professional responsibility
Senior residency
Service / education balance
Resident supervision
Clearly defined role of each site / rotation
Educational environment

36. B3 – Structure and Organization “Pitfalls”
Graded responsibility absent
Service/education imbalance
Service provision by residents should have a defined educational component including evaluation
Educational environment poor

37. B4 – Resources Number of teaching faculty
There must be sufficient resources –
Specialty-specific components as identified by
the Specialty Committee.
Number of teaching faculty
Number of variety of patients, specimens and procedures
Technical resources
Ambulatory/ emergency /community resources/experiences
Educational

38. B4 – Resources “Pitfalls”
Insufficient faculty for teaching/supervision
Insufficient clinical/technical resources
Infrastructure inadequate

39. B5 – Clinical, Academic and Scholarly Content of Program
The clinical, academic and scholarly content of the program must prepare residents to fulfill all the Roles of the specialist.
Educational program
Organized curriculum
Content specific areas defined by Specialty Committee
CanMEDS Roles
Teaching of the individual competencies

40. B5 – Clinical, Academic and Scholarly Content of Program “Pitfalls”
Organized academic curriculum lacking or entirely resident driven
Poor attendance by residents and faculty
Teaching of essential CanMEDS Roles missing
Role modeling is the only teaching modality

41. B6 – Assessment of Resident Performance
There must be mechanisms in place to ensure the systematic collection and interpretation of assessment data on each resident.
Based on objectives
Include multiple assessment techniques
Regular, timely, formal
Face-to-face

42. B6 – Assessment of Resident Performance “Pitfalls”
Mechanism to monitor, promote, remediate residents lacking
Formative feedback not provided and/or documented
Assessments not timely, not face to face
Summative assessment (ITER) inconsistent with formative feedback, unclearly documents concerns/challenges

43. How to Prepare for the Onsite Survey
Read the pre-survey questionnaire
Understand the standards

44. Conjoint Residency Education Accreditation System Reform

45. The Canadian Residency Accreditation Consortium (CanRAC)
Collaborative initiative between the Royal College, CFPC, and CMQ
New system of residency accreditation:
21st century best practices in accreditation;
Digitized; and,
Aligned with the shift towards competency-based medical education (CBME). 46
11/14/2018

46. 47

47. Shadow Surveyors Objective Role
To leverage the knowledge and expertise of experienced surveyors acting as “shadow surveyors” at an onsite survey and collect the necessary feedback on the new standards and process in accordance to the accreditation reform objectives
Role
Evaluate and validate new standards and process during an onsite survey visit
An “overlay” to the visit for selected programs or the institution to test the process with no impact to accreditation status / recommendations or decisions 48

48. Shadow Surveyor Code of Conduct
The shadow surveyor:
will act strictly as an observer during the onsite visit
will not engage in the discussions at various meetings, will not ask questions or seek clarification
will not influence in any way the recommendation of the surveyor and the survey team during the evening discussions
will not have voting privileges on the program‘s accreditation status recommendation 49

49. University of Alberta Onsite Survey
November 26 to December 1, 2017