Presentation is loading. Please wait.

Presentation is loading. Please wait.

Steven Kane, MD David Robinson, MD Atlanta Medical Center.

Published byCarlton Denning Modified over 9 years ago

Similar presentations

Presentation on theme: "Steven Kane, MD David Robinson, MD Atlanta Medical Center."— Presentation transcript:

1 Steven Kane, MD David Robinson, MD Atlanta Medical Center

2  Core Competencies  Prior USMLEs  Interviews  Faculty Evaluations  Intra-operative observation  Clinical observation  Surgical Skills lab  Peer to Peer  Pre medical experience  Ancillary staff evaluations  OITE  Milestones

3 9.63

4

5  “I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description and perhaps I could never succeed in intelligibly doing so. But I know it when I see it……” 1964 Potter Stewart

6  Carmichael, Kelly D. MD; Westmoreland, James B. MD; Thomas, John A. MD; Patterson, Rita M. PhD  May 2005, Southern Medical Journal  The only statistically significant correlations to OITE scores were USMLE step I performance (220) and marital status (married)

7  Paul J. Dougherty MD; Norman Walter MD; Peter Schilling MD; Soheil Najibi MD, PhD; and Harry Herkowitz MD  March 2010, Clinical Orthopaedics and Related Research  Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination  Correlation coefficient of 0.53 between the USMLE Step 1 and ABOS Part I  Average correlation coefficient of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I

8  KLEIN, GREGG R. MD; AUSTIN, MATTHEW S. MD; RANDOLPH, SUSAN; SHARKEY, PETER F. MD; HILIBRAND, ALAN S. MD  May 2004, The Journal of Bone and Joint Surgery-American Volume  Correlation coefficient for the USMLE Step-I score and the ABOS Part-I percentile score was 0.38 (p = 0.002)  Correlation coefficients were also calculated for the OITE percentile scores for years 3 (r = 0.52, p < 0.0001), 4 (r = 0.49, p < 0.0001), and 5 (r = 0.69, p < 0.0001).  There was a high risk of failure (63%; five of eight failed) on the ABOS Part-I examination when a resident scored below the twenty-ninth percentile for PGY-3 and below the twentieth percentile for PGY-5

9  5 programs evaluated 176 current and former residents evaluated  Chairs or PD’s asked to rate overall competence and surgical competence along with the 6 core competencies while “modestly blinded” to USMLE, OITE, and ABOS percentiles

10

11 A statistical cross comparison of USMLE 1, USMLE 2, OITE 1-5 percentiles, 6 Core Competencies, ABOS percentiles, Surgical Competency and Overall Competency upon graduation was compiled.

12

13 Variableby VariableN Spearman ρ Prob> │ρ│ ABOS Part 1 PercentileOITE 1250.1700.0417 ABOS Part 1 PercentileOITE 2650.478<.0001 ABOS Part 1 PercentileOITE 3650.2960.017 ABOS Part 1 PercentileOITE 4620.2510.049 ABOS Part 1 PercentileOITE 5600.463<.0001

14 Correlation with ABOS Survey CategoryNSpearman ρ Prob> │ρ│ Patient Care650.180.1501 Medical Knowledge650.54<.0001 Practice-Based Learning and Improvement650.280.0261 Interpersonal and Communication Skills650.270.0300 Professionalism640.280.0266 Systems-Based Practice650.270.0283 Surgical Competency650.370.0026 Overall Competency650.360.0035

15 Survey/Score CategoryScore CategoryRSp - Value Medical KnowledgeABOS %tile0.548650.000002 OITE %tile YIT-5ABOS %tile0.451600.000303 OITE %tile YIT-2ABOS %tile0.423650.000446 Surgical CompetencyABOS %tile0.390650.001321 Medical KnowledgeOITE %tile YIT-20.3851560.000001 Overall CompetencyABOS %tile0.370650.002415 Medical KnowledgeOITE %tile YIT-30.3631540.000004 Medical KnowledgeOITE %tile YIT-50.3371170.000199 OTIE %tile YIT-3ABOS %tile0.317650.010091 USMLE Step 2ABOS %tile0.299230.166287 ABOS was more correlative with surgical and overall competency than OITE (medium correlation)

16 Survey/Score CategoryScore CategoryRSp - Value PBL & ImprovementABOS %tile0.298650.015810 Systems-Based PracticeABOS %tile0.290650.019020 ProfessionalismUSMLE Step 20.298530.035895 Interper. & Comm. SkillsUSMLE Step 20.288530.036435 Surgical CompetencyOITE %tile YIT-20.2811560.000382 Interper. & Comm. SkillsABOS %tile0.272650.028624 Medical KnowledgeOITE %tile YIT-40.2601370.002153 Surgical CompetencyOITE %tile YIT-30.252620.048205 OITE %tile YIT-4ABOS %tile0.252620.048205 Overall CompetencyOITE %tile YIT-30.2341540.003492 OITE YIT- 4 Appears to be a throw away year

17 Survey/Score CategoryScore CategoryRSp - Value Surgical CompetencyOITE %tile YIT-10.155890.1464 Surgical CompetencyOITE %tile YIT-20.2811560.000382 Surgical CompetencyOITE %tile YIT-30.2591540.001167 Surgical CompetencyOITE %tile YIT-40.1311370.125677 Surgical CompetencyOITE %tile YIT-50.1921170.038341 Surgical CompetencyUSMLE Step 10.0531310.547163 Surgical CompetencyUSMLE Step 20.018530.900569 Surgical CompetencyABOS %tile0.39650.001321 The only objective score to correlate with surgical competency was ABOS I percentile (medium correlation)

18 Survey/Score CategoryScore CategoryRSp - Value Overall CompetencyOITE %tile YIT-10.136890.202862 Overall CompetencyOITE %tile YIT-20.2311560.003719 Overall CompetencyOITE %tile YIT-30.2341540.003492 Overall CompetencyOITE %tile YIT-40.1041370.224354 Overall CompetencyOITE %tile YIT-50.1331170.154008 Overall Competency has no correlation with OITE YIT 1-5

19  Our “mildly blinded” study of 5 programs, objective measurements have very little correlation with a program director or chair’s opinion of a resident’s overall competency, core competencies outside of medical knowledge or surgical competency upon graduation.  The one exception is the ABOS percentile and surgical competency which is mildly correlative.

20 “When given a choice on who will operate on me or my children, with all other qualities being equal, I would choose the physician with the higher scores….” Steven Frick. June 2012

21

Download ppt "Steven Kane, MD David Robinson, MD Atlanta Medical Center."

Similar presentations

The ERAS Application Can Predict ACGME Competency-Based Surgical Resident Performance Amy M. Tolan MD *, Amy H. Kaji MD PhD *, Chi Quach †, O.Joe Hines.

The ERAS Application Can Predict ACGME Competency-Based Surgical Resident Performance Amy M. Tolan MD *, Amy H. Kaji MD PhD *, Chi Quach †, O.Joe Hines.
Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD.

Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD.
American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants Who Will Pass the Examinations on the First Attempt? John L. Shellito,

American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants Who Will Pass the Examinations on the First Attempt? John L. Shellito,
Charles Holden, MD- Associate Program Director Thomas Hartranft, MD – Program Director Darrell Spurlock, Jr. PhD – Director of Research Mount Carmel Health.

Charles Holden, MD- Associate Program Director Thomas Hartranft, MD – Program Director Darrell Spurlock, Jr. PhD – Director of Research Mount Carmel Health.
Daniel Farkas, Kamal Nagpal, Ernesto Curras, Ajay Shah, John Cosgrove

Daniel Farkas, Kamal Nagpal, Ernesto Curras, Ajay Shah, John Cosgrove
 Alterman DM, Jones TM, Daley BJ, Goldman MH Department of Surgery.

 Alterman DM, Jones TM, Daley BJ, Goldman MH Department of Surgery.
Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education.

Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education Teaching Quality Improvement: A Needs Assessment for OBGYN Resident Education.
Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.

Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.
Comparison of Surgical Residency Applicants from U.S. Medical Schools with U.S.-Born and Foreign- Born International Medical School Graduates Paul J. Schenarts,

Comparison of Surgical Residency Applicants from U.S. Medical Schools with U.S.-Born and Foreign- Born International Medical School Graduates Paul J. Schenarts,
Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,

Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,
Performance on Brief Practice Exam Identifies residents at Risk for Poor ABSITE and ABS Qualifying Exam Performance Michael Corneille MD, Ross Willis PhD,

Performance on Brief Practice Exam Identifies residents at Risk for Poor ABSITE and ABS Qualifying Exam Performance Michael Corneille MD, Ross Willis PhD,
Quantifying of avascular necrosis of femoral head The clinical problem Determining the risk of femoral head collapse in a patient with AVNFH.

Quantifying of avascular necrosis of femoral head The clinical problem Determining the risk of femoral head collapse in a patient with AVNFH.
Printed by www.postersession.com Quantitative Assessment of Interviewing Competencies David A Goldberg MD, Steven P Reidbord MD, Dongmei Yue MD California.

Printed by Quantitative Assessment of Interviewing Competencies David A Goldberg MD, Steven P Reidbord MD, Dongmei Yue MD California.
Assessing clinical judgment using the script concordance test: The importance of using specialty-specific experts to develop the scoring key Petrucci AM.

Assessing clinical judgment using the script concordance test: The importance of using specialty-specific experts to develop the scoring key Petrucci AM.
... Our Departmental Family Tree Department of Otolaryngology – Head and Neck Surgery at The Ohio State University Wexner Medical Center X 1.

... Our Departmental Family Tree Department of Otolaryngology – Head and Neck Surgery at The Ohio State University Wexner Medical Center X 1.
Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success? Thomas W. Biester MS, Jonathan D. Rubright.

Does success on the American Board of Surgery Qualifying Examination guarantee Certifying Examination success? Thomas W. Biester MS, Jonathan D. Rubright.
Assessment of Emergency Medicine Residents’ Bedside Communication Skills: A Survey of Emergency Department Patients Amanda Keller York College of PA Biology.

Assessment of Emergency Medicine Residents’ Bedside Communication Skills: A Survey of Emergency Department Patients Amanda Keller York College of PA Biology.
A Survey of Selection Criteria in General Surgery Residency Programs George Makdisi MD 1 Tetsuya Takeuchi MD 2 James Rucinski MD 2 Leslie Wise MD 2 1 Department.

A Survey of Selection Criteria in General Surgery Residency Programs George Makdisi MD 1 Tetsuya Takeuchi MD 2 James Rucinski MD 2 Leslie Wise MD 2 1 Department.
Orthopaedic Education—Are We Attracting the Best and the Brightest? by Sanford E. Emery, James P. Waddell, Andrea E. Waddell, Michael McCaslin, and Kevin.

Orthopaedic Education—Are We Attracting the Best and the Brightest? by Sanford E. Emery, James P. Waddell, Andrea E. Waddell, Michael McCaslin, and Kevin.
Jonathan Fryer, Noreen Corcoran, Brian George, Ed Wang, Deb DaRosa Northwestern University Feinberg School of Medicine, Chicago Illinois.

Jonathan Fryer, Noreen Corcoran, Brian George, Ed Wang, Deb DaRosa Northwestern University Feinberg School of Medicine, Chicago Illinois.

Similar presentations

Ads by Google