US MLE United States Medical Licensing Examination u TM USMLE Step 2 Clinical Skills ASPE Webinar December 10, 2003 Richard Hawkins, Ann King National Board of Medical Examiners

US MLE United States Medical Licensing Examination u TM Overview Introduction: USMLE Step 2 CS Step 2 CS design (content, structure, scoring) General policies Exam logistics (dates, exam sites) Effects on medical education

US MLE United States Medical Licensing Examination u TM USMLE Purpose Serves the licensure process –Independent verification of competence –Educational experiences vary Co-sponsored by FSMB and NBME –Step 2 CS  collaboration with ECFMG Includes essential attributes for practice –Cognitive and clinical skills required for safe and effective patient care

US MLE United States Medical Licensing Examination u TM USMLE Step 2 CS “CSE” – enhancement of Step 2 USMLE Step 2 – components: –Clinical knowledge (CK) –Clinical skills (CS) Steps 1 & 2 prerequisite for Step 3

US MLE United States Medical Licensing Examination u TM Purpose: USMLE Step 2 CS Step 2 Clinical Skills assesses whether an examinee has the clinical skills essential to the safe and effective practice of medicine, with a focus on those clinical presentations that are common to patient care under supervision…

US MLE United States Medical Licensing Examination u TM Step 2 CS Blueprint Defines content categories Individual test form: –Adequate sampling of content domain –Comparable content between test forms

US MLE United States Medical Licensing Examination u TM CSE Blueprint: Content Categories Common and important medical problems / patient presentations Acuity Age Gender Race / ethnicity

US MLE United States Medical Licensing Examination u TM Case Content Cardiovascular Respiratory Gastrointestinal Musculoskeletal Constitutional Neurological Psychiatric Genitourinary Women’s health Other Test Form Patient age Age less than 18 Age 18 – 44 Age 45 – 64 Age 65 + Patient Gender Male Female Case Acuity Acute Subacute / Chronic

US MLE United States Medical Licensing Examination u TM Step 2 CS Structure 12 patient encounters 15 min. per encounter / 10 min. for patient note Each encounter: –Elicit pertinent history –Perform appropriate physical examination –Communicate effectively –Document: »Findings from the history and physical »Diagnostic impression / Further work-up

US MLE United States Medical Licensing Examination u TM Step 2 CS Structure Other station formats: –Third party interviews –Telephone encounters –Physical examination stations Future formats: –Difficult or sensitive communication issues –Synthetic models and mannequins

US MLE United States Medical Licensing Examination u TM Step 2 CS Structure: Score Components Integrated Clinical Encounter (ICE) –Data gathering: History and physical exam –Patient Note Communication / Interpersonal Skills (CIS) –Gathering/sharing information, manner, rapport Spoken English Proficiency (SEP) –Listener effort, examinee pronunciation / word choice

US MLE United States Medical Licensing Examination u TM Scoring - 2 In order to Pass Step 2 CS, examinees required to pass all three components: ICE, CIS, SEP Those who fail and then retake: –Reassessed in all three components –Must pass all three

US MLE United States Medical Licensing Examination u TM Standard Setting Step 2 CS system will be comparable to that used for the rest of USMLE Decision ultimately made by the Step 2 Committee –Will establish minimum passing point for ICE, CIS, and SEP separately –Will consider data from multiple sources »Survey of stakeholders »Independent review of content/encounters »Performance/reliability data –Closely monitor

US MLE United States Medical Licensing Examination u TM Score Reporting Standard Setting  summer – fall 2004 –Adequate numbers of examinees required Initial score reports  fall 2004

US MLE United States Medical Licensing Examination u TM Score Reporting - 2 To Examinees –Overall pass/fail outcomes –Failing examinees: graphical representation of relative strengths and weaknesses (similar to current USMLE reports)

US MLE United States Medical Licensing Examination u TM USMLE Step 2 Clinical Skills P/F Outcome Total Test FAIL Examination Components Integrated Clinical Encounter Pass Communication/Interpersonal Skills Fail Spoken English Proficiency Pass Sample Performance Report

US MLE United States Medical Licensing Examination u TM Performance Profile Integrated Clinical Encounter Data-gathering Patient note Communication / Interpersonal Skiills Spoken English Proficiency Lower Performance Borderline Performance Higher Performance XXXXXXXXXXXXXXXXXX xxxxxxxxxxxxxxxxxxxxxxxxxxxx

US MLE United States Medical Licensing Examination u TM Score Reporting - 3 To Schools –Individual student pass/fail outcomes »If student permits –Group summary reported yearly »Including CS components

US MLE United States Medical Licensing Examination u TM General Policies (for US/Canadian Students) Who must take Step 2 CS? –Those who graduate in 2005 or later –Earlier grads if Step 2 CK not passed by June 30, 2005 What are the eligibility requirements? (no change) –Student/grad of LCME- or AOA-accredited school Is there a required sequence? (no change) –Step 1, Step 2 CK, Step2 CS in any order Rules for repeaters (no change) –Wait 60 days between attempts, no more than three times in 12 month period, no USMLE imposed limit on number of takes (but the licensing authority may have limits)

US MLE United States Medical Licensing Examination u TM Exam Logistics First exam administered in June 2004 Collaboration with ECFMG –Step 2 CS replaces ECFMG CSA for IMGs Delivered at 5 regional test centers

US MLE United States Medical Licensing Examination u TM Test Centers Current ECFMG CSA test sites »Philadelphia »Atlanta Under Construction »Chicago »Los Angeles »Houston

US MLE United States Medical Licensing Examination u TM Exam Administration Schedule Centers open in sequence –First Center – June 2004 –All Centers open – September 2004 Exact dates and sequence TBA Capacity –3 examinations/day (33 examinees) –Up to 7 days/week Equal access to all Centers by US, Canadian and international examinees

US MLE United States Medical Licensing Examination u TM Pre-Implementation Pilot Exams Purpose –Realistic experience for SPs –Test AV and IT systems –Train support staff Scope –6-8 weeks prior to implementation at each site –Approximately 400 examinees at each site

US MLE United States Medical Licensing Examination u TM Participation in Pilots Open to all registered examinees –Mechanisms for allocating slots under development Mock examinations very similar to actual CS –Experience case design, SP interaction, examination logistics, Center –Limited feedback, not meant to be predictive Mix of US, Canadian and International Examinees Scheduling details TBA

US MLE United States Medical Licensing Examination u TM Registration & Scheduling Registration and scheduling will become available concurrently in early January Students and graduates of US and Canadian medical schools will register using the NBME Interactive Website for Applicants and Examinees (telephone registration available) Detailed information on registration, scheduling and fee structure on USMLE Website

US MLE United States Medical Licensing Examination u TM CSE: Effect on Medical Education Most significant impact at UME level Consequence of: –Effect of assessment on education –Placement of CSE in USMLE series

US MLE United States Medical Licensing Examination u TM CSE: Effect on Medical Education Accountability for clinical skills acquisition (public and students) Identifies need for national consensus on objectives for clinical skills teaching and assessment

US MLE United States Medical Licensing Examination u TM CSE: Effect on Medical Education Central role of the patient in medical education Clinical skills – curricular reform / emphasis Implications for resource allocation Faculty participation –Enhanced faculty interest and involvement –Support for faculty development

US MLE United States Medical Licensing Examination u TM CSE: Effect on Medical Education Challenge for Medical Educators: Ensure minimal standard (“teach to the test”), or… Think beyond minimum standards Don’t miss the opportunity to: –Fully explore clinical skills teaching / assessment –Inspire curiosity and enthusiasm for learning around patient encounters –Facilitate development of lifelong learning skills related to patient encounters…

US MLE United States Medical Licensing Examination u TM Standardized Patient Educators Will assume a more significant role Valued as experts: SP, clinical skills Role as a consultant More influence of educational programs With recognition, comes responsibility

US MLE United States Medical Licensing Examination u TM CSE: Other Implications For Residency Program Directors: –Improved applicant selection –Decreased time with problem residents –Better foundation on which to build Continued exploration and application of SPs and other simulation methods: –Advantages: faculty sparing, patient (and student) safety, ensure broad exposure… –Applications across continuum of education and practice

US MLE United States Medical Licensing Examination u TM Implications for the Profession For the FSMB, NBME and ECFMG: Values statement Generation of doctors will understand the relevance and importance of clinical skills Relationship between NBME and ECFMG and “Medical Education” will continue to grow Implications for the public are significant