The CORD-EM Speaker Evaluation Form for Medical Conference Planners 1 Andrew W Phillips MD MEd, 2 David Diller MD, 3 Sarah R Williams MD, 4 Yoon Soo Park PhD, 5 Jonathan Fisher MD, 6 Kevin Biese MD MAT, 7 Jacob Ufberg MD 1 Stanford University Dept of Emergency Medicine and Division of Critical Care Medicine, 2 Oregon Health and Sciences University Dept of Emergency Medicine, 3 Stanford University Dept of Emergency Medicine, 4 University of Illinois at Chicago Department of Medical Education, 5 Maricopa Medical Center Department of Emergency Medicine, 6 University of North Carolina Department of Emergency Medicine, and 7 Temple University Department of Emergency Medicine INTRODUCTION The CORD-EM Speaker METHODS CORD evaluators for CS form; 4 evaluators for CORD-EM form Content and construct validity sought for both forms Internal-consistency reliability: Cronbach’s alpha Inter-rater reliability: intraclass correlation (ICC) Speaker evaluations are an essential part of medical conferences and are required to provide CME. Existing forms are lengthy and focus on speaker improvement. A need remains for a form that can reliably separate stronger speakers by validated criteria. The Competent Speaker 3 Domains Knowledge Behavior Motivation 8 Competencies 1.5 hrs training Extensive prior validity and reliability evidence in public speaking competitions and courses 3 Domains Knowledge Behavior Motivation 3 Competencies NO training

1 Andrew W Phillips MD MEd, 2 David Diller MD, 3 Sarah R Williams MD, 4 Yoon Soo Park PhD, 5 Jonathan Fisher MD, 6 Kevin Biese MD MAT, 7 Jacob Ufberg MD 1 Stanford University Dept of Emergency Medicine and Division of Critical Care Medicine, 2 Oregon Health and Sciences University Dept of Emergency Medicine, 3 Stanford University Dept of Emergency Medicine, 4 University of Illinois at Chicago Department of Medical Education, 5 Maricopa Medical Center Department of Emergency Medicine, 6 University of North Carolina Department of Emergency Medicine, and 7 Temple University Department of Emergency Medicine The Competent Speaker FormThe CORD-EM Form The CORD-EM Speaker Evaluation Form for Medical Conference Planners

1 Andrew W Phillips MD MEd, 2 David Diller MD, 3 Sarah R Williams MD, 4 Yoon Soo Park PhD, 5 Jonathan Fisher MD, 6 Kevin Biese MD MAT, 7 Jacob Ufberg MD 1 Stanford University Dept of Emergency Medicine and Division of Critical Care Medicine, 2 Oregon Health and Sciences University Dept of Emergency Medicine, 3 Stanford University Dept of Emergency Medicine, 4 University of Illinois at Chicago Department of Medical Education, 5 Maricopa Medical Center Department of Emergency Medicine, 6 University of North Carolina Department of Emergency Medicine, and 7 Temple University Department of Emergency Medicine Competent Speaker Form Results Item Item Discrimination Overall.466 The CORD-EM Speaker Evaluation Form for Medical Conference Planners Item Item Discrimination Overall.900 CORD-EM Form Results DESCRIPTORS 11 independent speakers evaluated by 2 independent raters for 22 total evaluations Mean score 20.4 (2.8); min=13, max=24 VALIDITY Item discrimination ranged poor to strong (Table) RELIABILITY Moderately low internal- consistency reliability Poor overall inter-rater reliability despite 1.5 hrs of training DESCRIPTORS 21 independent speakers evaluated by 4 independent raters for 46 total evaluations Mean score 8.1(1.2); min=6, max=9 VALIDITY Excellent item discrimination (Table) Theoretical relationships supported quantitatively RELIABILITY Excellent internal consistency reliability Good overall inter-rater reliability withOUT preparatory training

1 Andrew W Phillips MD MEd, 2 David Diller MD, 3 Sarah R Williams MD, 4 Yoon Soo Park PhD, 5 Jonathan Fisher MD, 6 Kevin Biese MD MAT, 7 Jacob Ufberg MD 1 Stanford University Dept of Emergency Medicine and Division of Critical Care Medicine, 2 Oregon Health and Sciences University Dept of Emergency Medicine, 3 Stanford University Dept of Emergency Medicine, 4 University of Illinois at Chicago Department of Medical Education, 5 Maricopa Medical Center Department of Emergency Medicine, 6 University of North Carolina Department of Emergency Medicine, and 7 Temple University Department of Emergency Medicine REFERENCES CONCLUSIONS Morreale S. “ The Competent Speaker”: Development of a Communication-Competency Based Speech Evaluation Form and Manual. 2nd ed. Morreale S, Moore M, Surges-Tatum D, Webster L, editors. National Communication Association; Wittich CM, Mauck KF, Mandrekar JN, Gluth KA, West CP, Litin SC, et al. Improving Participant Feedback to Continuing Medical Education Presenters in Internal Medicine: A Mixed-Methods Study. J Gen Intern Med. Springer; 2012 Apr 1;27(4):425–31. The CORD-EM Speaker Evaluation Form for Medical Conference Planners Despite prior validity evidence, the Competent Speaker form competencies did not appear well related in a CME setting. Internal consistency and inter-rater reliability were also not adequate in the CME setting. Notably this form also required preparatory training, which is impractical for attendees prior to rating speakers. The CORD-EM form demonstrated excellent content and construct validity that was further supported quantitatively with the item discrimination correlation. Two of the individual items fared poorly with regard to inter-rater reliability, but the scale overall, which would be used to make speaker decisions, was strong and appropriate for use in the CME setting. Future work should include rewording questions 1 and 2 to improve the inter-rater reliability. ACCME. The accreditation requirements and descriptions of the Accreditation Council for Continuing Medical Education (ACCME) [Internet]. Chicago, IL, USA: Accreditation Council for Continuing Medical Education; Available from: ocument_0.pdf Wood TJ, Marks M, Jabbour M. The development of a participant questionnaire to assess continuing medical education presentations. Med Educ Jun;39(6):568–72. The authors wish to thank Drs. Megan Fix, Damon Kuehl, Janis Tupesis, and Moshe Weizberg for their contributions to the reliability studies as evaluators. Single location and specialty. Medical educators with lecture expertise. Only two evaluators for the Competent Speaker form, new to CORD. Only four evaluators for the CORD-EM form. LIMITATIONS