1. Medical Licensure in the United States Assessment of Clinical Skills Ann C. Jobe, MD, MSN Executive Director Clinical Skills Evaluation Collaboration (CSEC)

2. Raising the Bar in Geriatrics Assessment 2010 Reynolds Grantee 9 th Annual Meeting October 25-27, 2010 St. Louis, Missouri (sample of slides from presentation)

3. Clinical Skills Evaluation Collaboration CSEC A Collaborative Partnership, established in 2003, between the Educational Commission for Foreign Medical Graduates (ECFMG) and the National Board of Medical Examiners (NBME)

4. A Look Back in Time USMLE - introduced in early 1990’s Single examination pathway for initial medical license (graduates of US and international medical schools) A national standardized series of exams to assure minimal competency

5. United States Medical Licensing Examination (USMLE) Each of the three Steps of the USMLE complements the others No Step can stand alone in the assessment of readiness for medical licensure.

6. United States Medical Licensing Examination (USMLE) Step 1 understanding and application of important concepts of the sciences basic to the practice of medicine Multiple choice exam Step 2 application of medical knowledge, skills, and understanding of clinical science essential for the provision of patient care under supervision Clinical Knowledge (CK) Multiple choice exam Clinical Skills (CS)

7. United States Medical Licensing Examination (USMLE) Step 2CS Standardized patients used to assess an examinee’s ability to gather information from patients, perform physical examinations, communicate their findings to patients and colleagues

8. United States Medical Licensing Examination (USMLE) Step 3 application of medical knowledge and understanding of biomedical and clinical science essential for the unsupervised practice of medicine Multiple choice exam and computerized case simulations (CCS)

9. Life Cycle of a Physician in the United States Step 1 Year 1 Year 2 Medical School Year 3 Year 4 Step 2 CK Step 2 CS NRMP Match Postgraduate Training 3-7 yrs Step 3 Fellowship Board Certification Exams Recertification q 7-10 yrs Practice Licensure Maintenance of Certification American Board of Medical Specialties (ABMS) Specialty Boards State Licensing Authorities United States Medical Licensing Examination (USMLE) Relicensure q 1-3 yrs Maintenance of Licensure

10. Miller’s Pyramid Action Performance Competence Knowledge KNOWS KNOWS HOW SHOWS HOW DOES

11. Kirkpatrick Criteria 4.Results Change in organizational practice Benefits to patients/clients 3.Behavior Transfer learning to workplace Learners apply new knowledge and skills 2.Learning Change attitudes/perceptions Change knowledge/skills 1.Reaction Customer satisfaction related to participation in educational activities

12. COMPETENCY “Core Competencies” Accreditation Council for Graduate Medical Education (ACGME) American Board of Medical Specialties (ABMS) Patient Care Knowledge Communication and Interpersonal Skills Professionalism Systems-Based Practice Practice-Based Learning and Improvement

13. Correlations among Step scores Step l Step 2 CK Step 2 CS Step 2 CK ~.65-------------- Step 2 CS Data-gathering Communication/IP skills Communication/IP skills Spoken English proficiency Spoken English proficiency Patient note Patient note.19.19.09.09.20.26.16.13.30----- Step 3 ~.50~.70N/A

14. USMLE Step 2 Clinical Skills Mastery of clinical and communication skills, as well as cognitive skills, by individuals seeking medical licensure is important to the protection of the public. (from USMLE Bulletin of Information)

15. Step 2 CS Examinees YEARTOTALUSMGsIMGs 200934,83718,98315,854 200835,22417,71117,513 200733,83217,71116,121 200632,84317,47315,132 200531,93917,67114,268

16. Step 2 CS Fees USMG – increase of $100(10%) over 6 year period –2004-2006: $975 –2007: $1,005 –2008: $1,025 –2009: $1,055 –2010: $1,075 IMG –2004-2009: $1,200 –2010: $1,295

17. USMLE Step 2 Clinical Skills The cases cover common and important situations that a physician is likely to encounter in clinics, doctors’ offices, emergency departments, and hospital settings in the United States.

18. Presentation categories Include, but are not limited to, cardiovascular, constitutional, gastrointestinal, genitourinary, musculoskeletal, neurological, psychiatric, respiratory, and women's health. Examinees will see cases from some, but not all, of these categories. The selection of cases is also guided by specifications relating to acuity, age, gender, and type of physical findings presented in each case.

19. Each Step 2 CS “session” includes 12 twenty-five minute patient encounters. 15 minutes with patient 10 minutes for patient note The examination lasts approximately 8 hours. Two breaks are provided: 1 st break is 30 minutes long (lunch) 2 nd break is 15 minutes long (snack).

20. Step 2 CS Components Communication and Interpersonal Skills (CIS) Spoken English Proficiency (SEP) Integrated Clinical Encounter (ICE) Data gathering (DG) History & PE Patient note (PN)

21. Step 2 CS Components USMLE Step 2 CS is a Pass/Fail examination Each of the three subcomponents (CIS, SEP, ICE) must be passed in a single administration in order to achieve a passing performance on Step 2 CS

22. Assessment of Communication and Interpersonal Skills (CIS) CIS performance is assessed by the standardized patients a global rating of these skills using a series of generic rating scales same CIS scale for all 12 encounters 3 sub-components: Information gathering (questioning skills) Information sharing Professional manner and rapport

23. Assessment of Communication and Interpersonal Skills (CIS) Questioning skills - examples include: use of open-ended questions, transitional statements, facilitating remarks avoidance of leading or multiple questions, repeat questions - unless for clarification, medical terms/jargon unless immediately defined, interruptions when the patient is talking accurately summarizing information from the patient

24. Assessment of Communication and Interpersonal Skills (CIS) Information-sharing skills - examples include: acknowledging patient issues/concerns and clearly responding with information avoidance of medical terms/jargon unless immediately defined clearly providing counseling when appropriate closure, including statements about what happens next

25. Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include: asking about expectations, feelings, and concerns of the patient support systems and impact of illness, with attempts to explore these areas showing consideration for patient comfort during the physical examination attention to cleanliness through hand washing or use of gloves

26. Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include: providing opportunity for the patient to express feelings and/or concerns encouraging additional questions or discussion making empathetic remarks concerning patient issues/concerns patient feel comfortable and respected during the encounter

27. Assessment of Spoken English Proficiency (SEP) SEP performance is assessed by the standardized patients using rating scales; same scale for all 12 encounters based upon frequency of pronunciation or word choice errors that affect comprehension amount of listener effort required to understand the examinee's questions and responses clarity of spoken English communication within the context of the doctor-patient encounter (e.g., pronunciation, word choice, and minimizing the need to repeat questions or statements)

28. Scoring of the Step 2 Clinical Skills Subcomponents The ICE subcomponent includes assessment of: Data gathering (DG) - patient information collected by history taking and physical examination Documentation (PN) - completion of a patient note summarizing the findings of the patient encounter, diagnostic impression, and initial patient work-up

29. Scoring of the Step 2 Clinical Skills Subcomponents Data gathering (DG) performance is assessed by the standardized patients using checklists developed by committees of clinicians and medical school clinical faculty checklists comprise the essential history and physical examination elements for each specific clinical encounter

30. Scoring of the Step 2 Clinical Skills Subcomponents The patient note is rated/scored by trained physician raters The patient note (PN) consists of three areas Medical History and Physical Examination Differential Diagnosis Diagnostic Workup

31. Scoring of the Step 2 Clinical Skills Subcomponents Scored holistically Relevant and correct information Congruency/consistency with specific case – based scoring guidelines Integration/synthesis of information Organization, coherence, cohesiveness, flow, legibility

32. Performance on Step 2 CS Failure rate for USMGs 2004-2005: 4% 2005-2006: 2% 2006-2007: 3% 2007-2008: 3% 2008-2009: 3% Failure rate for IMGs 2004-2005: 17% US Citizens: 11% Foreign Citizens: 18% 2005-2006: 15% 2006-2007: 23% 2007-2008: 28% 2008-2009: 27%

33. Performance on Step 2 CS Passing rate for USMGs (first takers) 2005-2006: CIS >99%; SEP >99%; ICE 98% 2006-2007: CIS 99%; SEP 100%; ICE 97% 2007-2008: CIS 99%; SEP >99%; ICE 98% 2008-2009: CIS 99%; SEP >99%; ICE 98%

34. Performance on Step 2 CS Passing rate for IMGs (first takers) 2005-2006: CIS 93%; SEP 98%; ICE 89% 2006-2007: CIS 87%; SEP 99%; ICE 85% 2007-2008: CIS 81%; SEP 92%; ICE 86% 2008-2009: CIS 84%; SEP 94%; ICE 84%

35. For more information about USMLE policies and procedures, and about specific exams – please go to: www.usmle.org