1. ABPN Task Force on Clinical Skills Verification Rater Training

2. Co-Chairs Karen Broquet Michael Jibson AADPRT Michael Jibson Nyapati Rau Richard Summers AAP Karen Broquet Joan Anzia AACAP Eugene Beresin Jeffrey Hunt AACDP William Lawson David Wilson ACP David Kaye Tony Rostain ABPN Larry Faulkner Barbara Schneidman APA Deborah Hales Sandra Sexson

3. Task Force Timetable June 2008 – ABPN convened large group of psychiatric and neurologic educators to review literature on direct observation of learners and set goals for educating faculty raters. Task Force on Clinical Skills Verification Rater Training formed, with members from major psychiatric educational organizations. September 2008 – Task Force met and outlined curriculum. Meeting summary and action plan, and curricular outline available via these links on AADPRT website.

4. CSV Rater Training Curriculum 5 short Modules Each with10-15 minute didactic portion Some with video vignettes Some with narrative/facilitator’s guide All with pre- and post test Designed to be suitable for group presentation or self-study

5. CSV Rater Training Modules 1. Introduction 2.Training Faculty Evaluators 3.Setting the Examination Climate and Giving Feedback 4.Strategies for Clinical Skills Assessment 5.CSV Performance Parameters

6. Clinical Skills Verification rater Training Module 1 Introduction Sandra Sexson, M.D. Tony Rostain, M.D., M.A.

7. Pre-Test Module 1 1.Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam? a.The resident must demonstrate competence in at least 5 evaluations b.The evaluations must be conducted by a Board-certified psychiatrist c.The successful evaluations must be achieved within 1 year of completing training

8. Pre-Test Module 1 2.Which of the following is acceptable as part of a clinical skills verification examination? a.videotaped interview b.standardized patient c.one-way mirror d.established patient of the resident

9. Pre-Test Module 1 3. The in-residency clinical skills verification exam provides an opportunity to: a.insure direct observation of resident interviews in training b.provide early assessment of clinical skills with time for remediation c.attempt to develop standards of assessment across residency training programs d.all of the above

10. The Context ABPN The last ABMS member organization to include a live patient exam in the certifying exam Standardized patient attempts unsuccessful The Concern The live patient exam unreliable Many candidates had trouble on interview section Anecdotal and RRC reports suggested that some residents were never or very infrequently observed by faculty interviewing patients

11. The Context The Challenge Would earlier, more formalized, assessment of this competency allow for remediation and improve the competency of our graduates?

12. What was our immediate response as psychiatry program directors? We took our lead from Nancy Reagan and initially said:

14. But then, we seized the opportunity And we are here today to join with Obama and say:

16. RRC Link—V.A.l.h h) In at least three evaluations with any patient type, in any clinical setting, and at any time during the program, residents must demonstrate satisfactory competence in: establishing an appropriate doctor/patient relationship, psychiatric interviewing, performing the mental status examination and in case presentation. Each of the three required evaluations must be conducted by an ABPN- certified psychiatrist, and at least two of the evaluations must be conducted by different ABPN-certified psychiatrists. Satisfactory demonstration of the competencies during the three required evaluations is required prior to completing the program

17. Where Are We? FIRST RESPONSE JUST SAY NO!! NOW!!!! YES, WE CAN!! Opportunity to: –Insure actual interviews in our programs –Attempt standardization of assessments across programs –Provide assessments of these clinical skills earlier in training to provide for potential for remediation –Develop methods for remediation of these specific competencies –Conduct education research in assessment and credentialing

18. What Is Required? PGY-1 2007 or PGY-2 2008 Passage of at least 3 CSV evaluations during training For residents who enter CAP in PGY IV, one CAP CSV can count toward the 3 Psychiatry CSVs Exams must be administered by board certified psychiatrists and no more than two exams can be done by any one psychiatrist Time – Minimum 45 minutes – 30 min interview + 15 min presentation, but can be longer

19. Patient Issues Unknown to resident but can be known to faculty (brief overview in morning report ok but no interview of any kind) Introductory patient material (intake forms) are ok to utilize No simulated or standardized patients No videotape reviews One-way mirrors are ok ABPN silent about whether done during regular work or in special sessions so ok either way

20. Evaluation Standards “Pass” at any time should be that of a competent practicing psychiatrist Must pass all 3 major components individually (physician-patient relationship; conduct of psychiatric interview, including mental status exam; case presentation All sub-components do not have to be passed in order for the component to be passed Because there is no limit to the number of times a resident can take the exam and no negatives for having to retake the exam, there should not be pressure to score the resident’s performance as acceptable on an evaluation

21. Evaluation Forms Require ABPN approval Approved forms are posted on ABPN website (ABPN.com) and AADPRT website Programs may submit their own form if they prefer for ABPN approval—procedure on ABPN website CSV forms maintained in resident files for review by ACGME site visitor and possibility of ABPN review

22. What Happens When Part II is No Longer Offered? Documentation of satisfactory performance on the clinical skills competencies must be verified by a current program director of an ACGME accredited Psychiatry program to the ABPN as a part of the credentialing process to take the single computerized ABPN Psychiatry exam.

23. Post-Test Module 1 1.Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam? a.The resident must demonstrate competence in at least 5 evaluations b.The evaluations must be conducted by a Board-certified psychiatrist c.The successful evaluations must be achieved within 1 year of completing training

24. Post-Test Module 1 1.Which of the following is correct regarding the Psychiatry RRC requirements for the clinical skills verification exam? b.The evaluations must be conducted by a Board-certified psychiatrist Each of the three required evaluations must be conducted by an ABPN-certified psychiatrist, and at least two of the evaluations must be conducted by different ABPN-certified psychiatrists. Satisfactory demonstration of the competencies during the three required evaluations is required prior to completing the program.

25. Post-Test Module 1 2.Which of the following is acceptable as part of a clinical skills verification examination? a.videotaped interview b.standardized patient c.one-way mirror d.established patient of the resident

26. Post-Test Module 1 2.Which of the following is acceptable as part of a clinical skills verification examination? c.one-way mirror The patient must be unknown to the resident. One-way mirrors are acceptable. Videotaped interviews or standardized patients are not.

27. Post-Test Module 1 3. The in-residency clinical skills verification exam provides an opportunity to: a.insure direct observation of resident interviews in training b.provide early assessment of clinical skills with time for remediation c.attempt to develop standards of assessment across residency training programs d.all of the above

28. Post-Test Module 1 3. The in-residency clinical skills verification exam provides an opportunity to: D.all of the above Assurance of direct observation of resident interview skills, early recognition of potential deficiencies with time for remediation, and the opportunity to develop standards of assessment across residency training programs are all possible outcomes