Domain Based Communication Skills Assessment Pilot For EU Doctors Annie M Cushing 1, Jean S Ker 2, Paul Kinnersley 3, Anthony N Warrens 1, Olwyn M R Westwood 1, Pascal McKeown 4

European Union Law: Article 53 of Directive 2005/36/EC ‘Persons benefitting from the recognition of professional qualification shall have a knowledge of languages necessary for practicing the profession in the host Member State’

Assessment is NOT a test of Clinical Knowledge Legal Framework: ‘Proportionate and necessary’ : 1.Applied in a non-discriminatory manner 2.Justified by overriding reasons based on the general interest 3.Suitable for securing the attainment of the objective which they pursue and 4.Not go beyond what is necessary to obtain that objective

Mis-communication is multi-factorial Language Issues Cultural differences in concept of roles, systems and professionalism Consultation style and Communication skills in doctor-patient relationship Cultural aspects of healthcare and team relationships Pilotti LS 2007, Watt D, 2007, Woodward-Kron R 2009, Trafford P 2010 (personal communications)

Development Process for Test of Applicants to the UK Foundation Programme Validity Tasks (GMC Tomorrow’s Doctors 2009, Scottish Doctor) Domain-based mark scheme (Communication) Setting the Standard – Pass mark Reliability Training: Assessors Clinicians and Simulated Patients Information for Candidates

BLUEPRINT TEMPLATE FOR POPULATING EXAM STATIONS CONTEXTS TASKS Routine Oral Emotional Challenge Diversity - culture, interpreter/ impairment TelephoneReading/ Writing PartnershipAcute Setting History Station 1 Station 15 Station 13Station 2Station 1 Drug History Explain Condition Station 5 Explain Treatment / Procedure Station 4Station 14 Examination/Procedure Station 15Station 3 Write Notes/Drug Chart Station 2 Station 7 Read Notes/Drug Chart Station 7 Team/Handover Station 6 Station 7 Station 7Station 6 &12 Team Receive Instructions Station 7 Bad News/Sensitive Issues/ Error/Complaint Station 14Station 5 3 rd Party/Interpreter Station 8Station 13 Shared Decision Making Station 9 Consent / Ethical Issues Station 10

DOMAINS A = English language skills 1. Listening comprehension: understands patient/colleague 2. Spoken English comprehension: patient/colleague understands 3. Reading comprehension: demonstrates understanding of written input 4. Written comprehension: writing understood to enable appropriate action B = Clinical communication 1. Opening the consultation: establishes relationship with patient 2. Professional /humanistic quality: continues to maintain the relationship 3. Information gathering: gathers enough information to understand the problem(s) 4. Examining the patient: patient is enabled to co-operate with examination 5. Explaining/negotiating/consenting: information is understood with appropriate patient involvement 6. Communication with colleagues/team members C = Patient ratings: Patient satisfied with doctor-patient relationship and understanding

OSCE Format 15 Station Exam, 10 Minutes each = 2hrs 30mins 5 th year Medical Students (n= 15 at each site)

Results (n = 58) Raw Percentage Mean = 81.4 Standard Deviation= 6.7 Cronbach’s alpha= 0.78 Standard Error of Measurement= 3.11 Standard Setting by the Borderline Regression Method Cut Score (pass mark)= Cut Score + 1 SEm= Cut Score + 2 SEm= 68.80

Cumulative Marks’ Curve (integrated frequency distribution) combined sites scores

Overall Marks vs no. of stations failed, overall cut score, confidence intervals and stations’ criterion

Means of Domains & Variance across All Stations A1-4 = Language, B1-5 = Consultation, B6 = Team, C= Patient Rating

Next Steps Pilot with Doctors qualifying from non-English Speaking Countries

Acknowledgements Sam Leinster Siobhan Fitzpatrick Pascal McKeownJonathan Silverman John Patterson Penny Trafford Lynne RusteckiTim Swanick Katie Petty-Saphon