1. University of Manitoba Pre-Survey Meeting with Program Administrators Date: July 3, 2013 Time: 2:00 to 4:00 p.m. Room: Pharmacy Apotex, Theatre # 264

2. Objectives of the Meeting Provide an overview of Accreditation Process Questionnaire and Appendices Schedule Standards Opportunity to network with colleagues

3. Who has been involved in a survey process? Last regular survey in 2008 External review Internal review

4. Is a process to: Improve quality of postgraduate medical education Provides a means for assessment of residency programs Assists program director in reviewing conduct of program Based on Standards Accreditation

5. Internal Reviews 1 2 3 4 5 6 Monitoring Six Year Survey Cycle

6. Pre-Survey Process Royal College Comments Questionnaires University Specialty Committee Questionnaires Questionnaires & Comments Program Director Comments Surveyor

7. First impression of the program by the surveyor is the questionnaire and appendices All sections of the questionnaire are complete Error-free vs typos Neat vs sloppy Clear descriptions vs confusing information Spell out abbreviations Questionnaire and Appendices

8. When submitting your questionnaire and appendices please format and name them as follows: Questionnaire Each individual section (i.e. each standard) is submitted as a separate PDF file The files are named as: General_Information.pdf, Standard_B1.pdf, Standard_B2.pdf, etc. Questionnaire and Appendices

9. Appendices For each appendix, the name of the appendix (e.g. Appendix A) is clearly indicated on the first page. Each individual appendix is submitted as a separate PDF file. The files are named as: Appendix_A.pdf, Appendix_B.pdf, etc. Any additional attachments are labelled/formatted as above with the name corresponding to the appendix name as listed in the General Information section of the PSQ (e.g. Appendix_K.pdf, Appendix_L.pdf, Appendix_M.pdf, etc.) Questionnaire and Appendices

10. Review – Length & Day Based on Number of residents in program Traveling to different sites Last schedule in 2008 Coordinated by the Royal College and PGME office Inform PGME office if you foresee a scheduling conflict PGME office will notify the program of the day of the review  Surgical Foundations will be reviewed on Monday morning. The Survey Schedule

11. Transportation Arranged by program Include details Name & contact info for person responsible Morning Pick-up at 07:45 a.m. At the Fairmont Winnipeg To and from different sites When required during review Afternoon Survey must end no later than 4:00 p.m. At the Fairmont Winnipeg The Survey Schedule

12. Sequence to follow Document review (30 min) Program director (75 min) Department head * (30 min) Residents * (per group of 20 - 60 min) Teaching faculty * (60 min) Residency Program Committee (60 min) Exit Meeting (15 min) Morning after the review concluded 07:30-07:45 a.m. at the Fairmont Winnipeg * Scheduled anytime after program director but before RPC The Survey Schedule

13. Document Review (30 min) Scheduled at beginning of schedule Available in meeting room Residency Program Committee minutes - last 6 years Resident assessment files If automated, arrange access to a computer, make sure that it is easy to navigate and retrieve Provide assistance/demo Remember surveyors have limited time to review the documents on-site The Survey Schedule

14. Program director (75 min) Scheduled after document review To discuss Overall view of program Evaluation of Standards Strengths, challenges and weaknesses Specialty Committee Comments − Sent by email to program director week prior to on-site visit The Survey Schedule

15. Department head (30 min) Scheduled anytime after program director but before RPC To discuss Support for program director and program Resources available to program The Survey Schedule

16. Residents (60 min) Scheduled anytime after program director but before RPC Groups of 20 residents − Can be organized in various groups  By postgrad year  By Junior & Senior years Encourage residents to: Complete the ‘confidential’ CAIR questionnaire Sent by CAIR Meet as a group to discuss strengths & challenges prior to visit Send reminders – date, time, locations The Survey Schedule

17. ALL residents invited to attend/participate in their resident meeting(s)? Residents who cannot attend Arrange link by video- or tele- conferencing Are visa trainees or (clinical) fellows invited to attend? Different funding / title across Canada For accreditation purposes only, a resident is a person who is following the same academic program/training as a resident eligible to write College exams – funding doesn’t matter ! The Survey Schedule

18. Teaching faculty (60 min) Scheduled anytime after program director but before RPC To discuss Involvement with residents Communication with program director Teaching faculty who sit on RPC need only attend RPC meeting The Survey Schedule

19. Tours (15 min) Optional Tours of wards/clinics not necessary Is there something special to showcase? MUST be scheduled before RPC The Survey Schedule

20. Residency Program Committee (60 min) Program director attends first half of meeting Resident representative(s) attend the meeting Smaller programs Often faculty all sit on RPC Can combine faculty & RPC together into one meeting MUST be scheduled as LAST meeting of review The Survey Schedule

21. Survey Team Discussion Your program will be discussed the evening of the day that your review ends Identify the strengths & weaknesses Provide a recommendation The Survey Schedule

22. Chair - Dr. Sarkis Meterissian –Responsible for general conduct of survey Deputy chair – Dr. Maureen Topps –Visits teaching sites / hospitals Surveyors Resident representatives – CAIR Regulatory authorities representative – FMRAC Teaching hospital representative – ACAHO The Survey Team

23. New terminology – June 2012 Approved by the Royal College, CFPC and CMQ. Recommendation Accredited program Follow-up: − Next regular survey − Progress report (Accreditation Committee) − Internal review − External review Accredited program on notice of intent to withdraw accreditation Follow-up: − External review The Survey Schedule

24. Exit Meeting (15 min) Morning after review 07:30 – 07:45 a.m. At the Fairmont Winnipeg Program director will be informed Survey team recommendation Strengths & weaknesses of program The Survey Schedule

25. BE PROACTIVE Start early Share the information with your program director Make a plan and set timelines Create and use your own checklist Consult with the PGME office In doubt: ask questions The Survey Schedule

26. Be available to assist the surveyors as needed. ‘Do Not Disturb – Meeting in Progress’ sign on door. Ensure that the surveyors are comfortable (i.e.: private room, water, a layout of the area, etc.) Arrange for beverages, healthy snacks for breaks and a lunch, when appropriate. Organize all the required documents in the room and ensure that there is a computer with internet access available to the reviewers. Retrieve all confidential documents from the room(s) at the end of the day. Day of the Survey

27. Final exit with University Friday, February 28, 2014 between 9:00 and 10:00 a.m. CFPC/Royal College Chairs present a summary of survey week Survey reports sent to PGME 6 to 8 weeks post-survey Program response sent to College Report any errors of fact only Accreditation Committee Decisions May/June 2014 Dean & postgraduate dean attend Appeal process is available After the Survey

28. “A” Standards Apply to University, specifically the PGME office “B” Standards Apply to EACH residency program Updated January 2011 “C” Standards Apply to Areas of Focused Competence (AFC) programs General Standards of Accreditation

29. Applicable to the University Reviewed by chair’s team (Team A) Schedule organized by PGME office Looking at Structure in place Training sites Liaison with hospitals “A” Standards

30. Applicable to ALL programs Reviewed by surveyors Looking at B1 Administrative Structure B2 Goals & Objectives B3 Structure and Organization B4 Resources B5 Clinical, Academic & Scholarly Content B6 Assessment of Resident http://www.royalcollege.ca/portal/page/portal/rc/common/documents/accreditation/ac creditation_blue_book_b_standards_e.pdf “B” Standards

31. There must be an appropriate administrative structure for each residency program. Program director Time & support Residency Program Committee Operation of the program Program & resident evaluations Appeal process Selection & promotions of residents Process for teaching & assessment of competencies Research Regular review of program Faculty assessments B1 – Administrative Structure

32. Program director autocratic Residency Program Committee dysfunctional –Unclear Terms of Reference (membership, tasks and responsibilities) Agenda and minutes poorly structured Poor attendance –Department head unduly influential –RPC is conducted as part of a Dept/Div meeting No resident voice B1 – Administrative Structure “Pitfalls”

33. There must be a clearly worded statement outlining the Goals & Objectives of the residency program. Rotation-specific Address all CanMEDS Roles Functional / used in: Planning Resident assessment Distributed to residents & faculty Reviewed regularly At least every 2 years B2 – Goals and Objectives

34. B2 – Goals & Objectives “Pitfalls” Missing CanMEDS roles in overall structure –Okay to have rotations in which all CanMEDS roles may not apply (research, certain electives) Goals and objectives not used by faculty/residents Goals and objectives dysfunctional – does not inform assessment Goals and objectives not reviewed regularly

35. There must be an organized program of rotations and other educational experiences to cover the educational requirements of the specialty. Increasing professional responsibility Senior residency Service / education balance Resident supervision Clearly defined role of each site / rotation Educational environment B3 – Structure & Organization

36. Graded responsibility absent Service/education imbalance –Service provision by residents should have a defined educational component including evaluation Educational environment poor B3 – Structure & Organization “Pitfalls”

37. There must be sufficient resources – Specialty-specific components as identified by the Specialty Committee. Number of teaching faculty Number of variety of patients, specimens and procedures Technical resources Ambulatory/ emergency /community resources/experiences Educational B4 - Resources

38. Insufficient faculty for teaching/ supervision Insufficient clinical/technical resources Infrastructure inadequate B4 – Resources “Pitfalls”

39. The clinical, academic and scholarly content of the program must prepare residents to fulfill all the Roles of the specialist. Educational program Organized curriculum - Content specific areas defined by Specialty Committee CanMEDS Roles Teaching of the individual competencies B5 – Clinical, Academic & Scholarly Content of Program

40. Organized academic curriculum lacking or entirely resident driven –Poor attendance by residents and faculty Teaching of essential CanMEDS roles missing Role modelling is the only teaching modality B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls”

41. Appendix ‘H’ Formal Academic Curriculum The surveyor will be verifying that the content of the academic curriculum covers all the of the CanMEDS Roles. B5 – Clinical, Academic & Scholarly Content of Program

42. There must be mechanisms in place to ensure the systematic collection and interpretation of assessment data on each resident. Based on objectives Include multiple assessment techniques Regular, timely, formal Face-to-face B6 – Assessment of Resident Performance

43. Mechanism to monitor, promote, remediate residents lacking Formative feedback not provided and/or documented Assessments not timely, not face to face Summative evaluation (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges B6 – Assessment of Resident Performance “Pitfalls”

44. University of Manitoba Regular on-site Survey February 23 to 28, 2014 44

45. accred@royalcollege.ca 613-730-6202 Office of Education Sarah Taber Assistant Director Education Strategy & Accreditation Educational Standards Unit Sylvie Lavoie Survey Coordinator Contact Information at the Royal College