1. Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success? Thomas W. Biester MS, Jonathan D. Rubright MS, Andrew T. Jones PhD, Mark A. Malangoni MD, FACS The American Board of Surgery, Philadelphia, PA

2. Board Certification Matters Important benchmark to achieve following completion of residency (Nahrwold DL, Surg, 1999) Associated with better outcomes in patient care (Wenthofer E, Med Educ 2009; Reid RO, Arch Intern Med, 2010) Often required to obtain or maintain hospital privileges

3. Mission of the American Board of Surgery Serves as an agent for protection of the public and for quality surgical care Certifies surgeons who have met education and training standards Sets standards for maintenance of certification

4. Process of Obtaining Board Certification Appropriate undergraduate medical education Successfully complete an ACGME- approved surgery residency program Obtain favorable summary evaluation from the program director Pass the ABS Qualifying Examination (QE) Pass the ABS Certifying Examination (CE)

5. Oral Exams “Bridge the gap” between the cognitive domain and behavioral evaluation of performance in practice 15/24 ABMS boards have an oral exam

6. Purposes Determine whether there is a correlation between passing the QE and passing the CE Ascertain whether there is a discrete QE score that would predict passing the CE

7. Design Examinees who passed the ABS QE from 2006-10 QE equated scores matched to 1 st CE pass/fail decisions Contingency table analysis Phi coefficient and point bi-serial for correlation ROC curve constructed p < 0.01

8. Results 4,385 examinees who passed the QE Mean QE score = 82.1 + 5.8 (range, 58-99) 92.3% passed the QE on the 1 st attempt 82.8% passed the CE on the 1 st attempt

9. Relationship between 1 st QE outcome and 1 st CE CE fail CE pass Total QE pass 639 (16%) 3407 (84%) 4046 (92%) QE fail 115 (34%) 224 (66%) 339 (8%) Total 754 (17%) 3631 (83%) 4385 Φ = 0.13, p < 0.001

10. 1 st QE scaled scores and CE pass/fail decisions Scale Score Fail Pass Total 55-64 4 (67%) 2 (33%) 6 65-69 37 (41%) 53 (59%) 90 70-74 74 (30%) 169 (70%) 243 75-79 286 (26%) 820 (74%) 1106 80-84 243 (17%) 1196 (83%) 1439 85-99 110 (7%) 1391 (93%) 1501 Total 754 (17%) 3631 (83%) 4385 CE QE r pb = 0.229, p < 0.001

12. 1 st QE scaled scores and successful certification (2001-05) Scale Score Not certified Certified Total 50-64 23 (96%) 1 (4%) 24 65-69 26 (54%) 22 (46%) 48 70-74 64 (33%) 133 (68%) 197 75-79 93 (17%) 442 (83%) 535 80-84 57 (5%)1054 (95%) 1111 85-99 38 (2%)1487 (98%) 1525 Total 313 (6%)4776 (94%) 5089 r pb = 0.339, p < 0.001

13. Summary The correlation between the 1 st QE score and CE pass/fail rate is low Failure on the 1 st QE doubles the risk of CE failure Excellent correlation between 1 st QE score and eventual board certification

14. Conclusions These exams measure different aptitudes (knowledge vs. application) QE scores and CE results correlate well only at the extremes There is no discrete QE score that predicts passing the CE on the 1 st attempt

15. Thank you