Using Objective Structured Clinical Exams (OSCEs) to Evaluate an Interdisciplinary Clinical Skills Teaching Curriculum Karen Maughan, MD, Eugene C Corbett Jr ,MD Veronica Michaelsen, MD, Nancy Payne, MD Elizabeth Bradley, PhD

AAMC Clinical Skills Education Objectives *Practice of professionalism Application of science to solve clinical problems *Development of patient engagement and relationship skills *Elution of clinical history *Performance of physical examination Selection and interpretation of clinical tests *Selection and performance of basic clinical procedures Management of clinical information Generation of prioritized differential diagnosis Practice of intervention, cure, and prevention Development of prognosis based on health risk and diagnosis *Knowledge of individual patient context in which medicine is practiced * Skills more appropriately assessed by OSCE than traditional knowledge-based tests www.aamc.org/meded/clinicalskills/

Clinical Skills Education in a Medical School Curriculum 1st year Physical exam skills Ear, nose, throat Lung Heart Upper/Lower extremity Vital Signs/Chest Exam Abdomen Neurological Eye Interviewing skills Basic interviewing skills Sexual History

Clinical Skills Education in a Medical School Curriculum 1st year Physical exam skills Ear, nose, throat Lung Heart Upper/Lower extremity Vital Signs/Chest Exam Abdomen Neurological Eye Interviewing skills Basic interviewing skills Sexual History 2nd year Physical exam skills Gynecologic Male GU Pulmonary Cardiac Neurological Ophthalmologic Rheumatologic Interviewing skills Breaking bad news

Clinical Skills Education in a Medical School Curriculum Basic Patient Care course Transition to clerkship IV placement Lumbar puncture Foley placement CXR interpretation EKG performance Central line placement Sterile field Arterial blood gas procurement

Clinical Skills Education in a Medical School Curriculum Family Medicine clerkship knee exam back exam ankle exam asthma, peak flow office derm derm procedures office gynecology geriatric falls assessment ear, nose, throat information mastery medicine at the point of care challenging patient encounter motivational interviewing

Clinical Skills Education in a Medical School Curriculum Ambulatory Internal Medicine clerkship ophthalmoscopic exam breast exam shoulder exam infectious disease, dx and rx EKG interpretation geriatric polypharmacy nephrology imaging and lab interpretation health economics rheumatology – hand and wrist evaluation CXR and PFTs

Clinical Skills Education in a Medical School Curriculum Pediatrics clerkship screening for abuse and neglect in children phone triage ear exam adolescent interview developmental delays and school failure

Observed Clinical Experiences in the Medical School Curriculum 1st year 7 Physical Exam OSCEs History Taking

Observed Clinical Experiences in the Medical School Curriculum 1st year 7 Physical Exam OSCEs History Taking 2nd year Complete history and physical

Observed Clinical Experiences in the Medical School Curriculum 1st year 7 Physical Exam OSCEs History Taking 2nd year Complete history and physical 3rd year Clinical skills passports for each clerkship CPX at end of clerkship year Clerkship Clinical Skills Education Project (OSCEs)

Despite a demonstrated interest in teaching clinical skills in different places in the curriculum ……

Despite a demonstrated interest in teaching clinical skills in different places in the curriculum …… there remains a need for a more coordinated teaching and assessment process

Teaching Clinical Skills Important

Teaching Clinical Skills Focused

Teaching Clinical Skills Feels good

Teaching Clinical Skills Rewarding

Barriers to Teaching Clinical Skills Time Money Evaluation bias Physician specialization Training in tertiary care environments Regulatory and payor influences Time- clinical teachers and learners have adequate time to teach and learn from patient interactions Money – as support for medical education diminishes, institutions give greater support for research and clinical endeavors Evaluation – traditional emphasis on standardized written assessment fosters a knowledge- rather than skill-based paradigm. It is also less expensive to use MCQ tests rather than observed skill assessment

Clinical Skills Workshops Development of an Interdisciplinary Clinical Skills Teaching Curriculum Formative Assessment Clinical Skills Workshops

Development of OSCEs for a Clinical Skills Assessment Identify clinical skills to be assessed Communication skills Physical exam skills Procedural skills Include skills from all clerkship areas

OSCE Development Communication OSCEs working with an interpreter motivational interviewing breaking bad news explaining a medical error assessing decisional capacity managing phone triage teaching a patient about dx of osteoporosis presenting a case after chart review screening for anxiety disorder

OSCE Development Physical exam OSCEs cardiac dysrrhythmia cardiac gallop geriatric gait assessment blood pressure measurement pediatric ear exam knee exam eye exam mental status exam pediatric hip exam shoulder exam

OSCE Development Procedural OSCEs airway management cervical sampling performance of EKG peak flow measurement information mastery aseptic technique IV insertion

Development of OSCEs for a Clinical Skills Assessment Recruit authors Identify and approach individuals with an interest in clinical teaching and expertise in the skills to be assessed Solicit commitment from department chairs to support project

Development of OSCEs for a Clinical Skills Assessment Recruit authors Identify and approach individuals with an interest in clinical teaching and expertise in the skills to be assessed Solicit commitment from department chairs to support project Guide authors Work with author to choose appropriate case Provide materials for case development

Development of OSCEs for a Clinical Skills Assessment Recruit authors Identify and approach individuals with an interest in clinical teaching and expertise in the skills to be assessed Solicit commitment from department chairs to support project Guide authors Work with author to choose appropriate case Provide materials for case development Lead editorial process Review case during development and provide feedback

Requirements for Conducting Clinical Skills Assessment Authors for OSCE cases Students (i.e. support of the school of medicine) Clinical Standardized Observers (SO) Standardized Patients (SP) and/or clinical models Faculty trainers for SOs and SPs Space to conduct assessment Technical support Someone who knows what to do with the data

Quantitative Results -

Clerkship Clinical Skills Education Project Comparison of average scores on OSCEs: students who have had workshops vs those who have not % of total possible points Knee workshops Pediatric ear Shoulder Eye Phone triage Infomastery

Clerkship Clinical Skills Education Project How often is there opportunity for skill practice in the clerkship? (student opinion asked at the time of the assessment)

Qualitative Results Extensive faculty participation Opportunities for teaching clinical skills at faculty, fellow, resident, student level Many faculty, fellows, residents, 4th year students have participated in multiple sessions Increased awareness of the importance of teaching clinical skills

Lessons Learned We like teaching clinical skills Need support of school of medicine for success Developing OSCEs is a consensus-driven, faculty-intense iterative process Students are appreciative of being taught clinical skills Many benefits to interdisciplinary involvement such as developing consistency and standardization in teaching Increased awareness of importance of clinical skills teaching at many levels Makes more explicit student clinical performance development in the curriculum Can link OSCEs to an interdisciplinary skills curriculum

Discussion