Steven Lin MD, Grace Chen Yu MD, and Erika Schillinger MD OSLER O’Connor-Stanford Leaders in Education Residency Program Steven Lin MD, Grace Chen Yu MD, and Erika Schillinger MD San Jose-O’Connor Family Medicine Residency Program and Stanford School of Medicine

Background Stanford University School of Medicine Total number of medical students: 468 Total number of family medicine faculty who teach: 13 1 of 10 “orphan” schools in the U.S. without a FM department O’Connor Family Medicine Residency Program Total number of residents: 8-8-8 Only FMRP affiliated with Stanford, established in 1977 Located in San Jose, California – 20 miles from Stanford

Background Problem Solution Lack of family medicine resident role models on campus Early student exposure to residents as teachers Lack of family medicine clinician-educators on faculty Teaching and leadership track to grow pipeline

Program Overview Joint initiative of the O’Connor FMRP and Stanford School of Medicine, created in 2010 Residents in OSLER complete a curriculum that resembles a faculty development fellowship

Program Goals To train a new generation of family medicine leaders in medical education To create and sustain a vibrant community of resident scholars, innovators, and leaders at O’Connor FMRP To advance primary care at Stanford University School of Medicine

Curriculum and Timeline

Year 1 Apply for OSLER track at midpoint of intern year Participate in monthly seminars and complete online leadership training Master presentation skills through direct observation and videotaping of journal club and case conferences Begin planning an educational scholarly project

Year 2 Teach medical students alongside faculty at Stanford Practice of Medicine Course Family Medicine Clerkship Standardized Patient Exams Master presentation skills through direct observation and videotaping of journal club and case conferences Design and conduct an educational scholarly project

Year 3 Teach medical students alongside faculty at Stanford Complete educational scholarly project and present at national meeting or submit to a peer-reviewed journal Master presentation skills through direct observation and videotaping of journal club and case conferences Finalize and review educational portfolio with mentor

Track Completion Checklist

Scheduling PROTECTED MONTHS FOR OSLER-RELATED ACTIVITIES YEAR 1 FCM Elective Selective YEAR 3 MODEL WEEK – FCM Monday Tuesday Wednesday Thursday Friday 8:30 – 12:30 Project/Tutorial 8:30 – 11:00 Clinic Seminar 8:30 – 5:00 Project 1:30 – 5:00 Community 12:15 – 5:00 Stanford MODEL WEEK – SELECTIVE/ELECTIVE Monday Tuesday Wednesday Thursday Friday 8:30 – 11:00 Clinic 8:30 – 5:00 Project 12:15 – 5:00 Stanford

Evaluation: OSLER Model MEASURES COMPETENCIES OBJECTIVES Does the resident set specific and measurable objectives? Are the resident’s methods of instruction appropriate? SETTING AND LEARNING CLIMATE Are the resident’s teaching sessions organized and efficient? Is it safe for learners to address their limitations? LEARNER-CENTERED TEACHING Does the resident’s teaching meet the learners’ needs? Does the resident’s teaching involve learners actively in their own learning and development? EVALUATION AND FEEDBACK Are the resident and trainee working as allies with common goals? Was the resident’s feedback timely, specific and non-judgmental? REFLECTION Does the resident encourage self-reflection? Is there a commitment for the resident and trainee to improve together?

Preliminary Results Today 11 of 24 residents (46%) at O’Connor FMRP are actively participating in OSLER Residents indicate high levels of satisfaction with the track’s overall value, impact on core skills, and effect on career trajectory Number of residents who completed the OSLER track: 6 Number of graduates invited to interview for faculty: 5 Number of graduates offered faculty appointments: 4

Residents’ Teaching Skills Comparison of self-evaluation data over time showed that residents gained more confidence in core teaching skills as they progressed through the track Using the Cleveland Clinic’s CTEI, residents reported statistically significant improvements in: Stimulating students to learn independently Organizing time to allow for both teaching and patient care Teaching cost-appropriate care

Residents’ Teaching Skills Clinical Teaching Skills Baseline After One Year P Value Stimulating students to learn independently 3.000 3.625 0.002 Organizing time to allow for both teaching and patient care 2.125 0.028 Teaching principles of cost-appropriate care 1.375 2.625 0.009

Residents’ Scholarly Work Scholarly work output increased significantly after the track was implemented For example, in 2014: 6 residents produced 8 abstracts, which were all accepted at national conferences 2 residents each published one peer-reviewed manuscript Among non-OSLER residents, no abstracts or papers were published in the same period

Residents’ Experiences

Medical Student Reflections “I spent a month at O’Connor for my family medicine clerkship and met residents who I could really relate to, especially the residents from Stanford who had chosen family medicine. Somehow this made my choice feel more validated. They were so smart and so passionate about their work.” Stanford Medical Student

Medical Student Reflections “Exposure to the O’Connor residents was very important because they gave me a sense of my next stage of training.” Stanford Medical Student “The OSLER program is a huge asset as it allows residents at O’Connor to teach medical students in Practice of Medicine course, and students get early exposure to family medicine by meeting skilled clinician-educators.” Stanford Medical Student

Faculty Reflections “The OSLER program’s impact on improving resident and student education, and the excitement it has brought to our entire residency program has been dramatic. We have seen a jump in interest in Family Medicine from Stanford medical students, as well as regional and national interest from medical students who have heard about the program from educators across the country.” Robert Norman, MD, Director of O’Connor FMRP

Faculty Reflections “This is a truly original and trailblazing program that has created a lot of excitement and interest in primary care education at Stanford – an institution that has not had a strong primary care presence historically. OSLER can serve as a pipeline for innovative family medicine clinician educators, something we need a lot more of. If this can work at Stanford, then it can work elsewhere.” Sang-ick Chang, MD, Assistant Dean for Clinical Affairs

Impact at Residency The residency program has seen an increased interest from applicants since the track was implemented Applicant quality – in terms of board scores, Dean’s letter scores, and the proportion of candidates with advanced degrees (e.g., MPH, PhD) – has increased significantly from pre-track to post-track years

Impact at Residency

Before track introduction After track introduction Impact at Residency Interviewee characteristics Before track introduction (2008-2011) n After track introduction (2012-2015) P value Mean USMLE Step 1 score 216 278 222 408 <0.0005 Mean USMLE Step 2 CK score 221 209 235 367 Mean Dean’s Letter score 3.22 283 3.48 412 Proportion of candidates with advanced degrees (e.g., MPH, PhD) 13% 37 21% 86 0.006

Post-interview Survey (2014-2015)

Post-interview Survey (2014-2015)

Post-interview Survey (2014-2015)

Impact at Stanford

Proportion of medical school graduates who matched into family medicine

Proportion of Stanford seniors who matched into primary care residencies

Conclusion A teaching and leadership track in residency advances teaching excellence, fosters student interest in primary care, and strengthens the pipeline for Family Medicine

Future Directions More data is needed to assess the effectiveness and reproducibility of our program If validated, this program may be replicated at other academic medical centers

THANK YOU! Steven Lin, MD stevenlin@stanford.edu Grace Chen Yu, MD graceyu1@stanford.edu Erika Schillinger, MD erikas@stanford.edu