1. State University of New York at Buffalo Primary Care Master Educator Program David Newberger, M.D. Elie Akl, M.D., Ph.D. * Denise McGuigan, M.S. Ed. Andrew Symons, M.D., M.S. Department of Family Medicine Department of General Internal Medicine *

2. A faculty development fellowship that enables primary care faculty to engage in scholarship related to medical education. Funded in part by a Faculty Development in Primary Care Grant from the U.S. Department of Health and Human Services. SUNY Primary Care Master Educator Program Family Medicine Pediatrics Internal Medicine

3. Introduction Problem: Many primary care faculty lack training in educational scholarship, which limits: ability to fully develop and share teaching innovations. recognition and reward via promotion and tenure.

4. Needs Assessment: U.S. ACGME residency program requirements. 2006 Academic Medicine : medical education fellowships, “teaching academies” 2005 survey of University at Buffalo (UB) Department of Family Medicine Faculty. 2006 survey of UB department chairs and faculty.

5. Current Medical Education Training for UB Faculty: Since 2006, 78 UB faculty have completed the Royal College of Physicians Educator Program. Two Education Initiatives:  Purpose  Starting conditions  Actions taken  Effectiveness  Conclusion

6. Opportunity: 2008 Title VII Faculty Development in Primary Care Grant Program: Master Educator Fellowships. Opportunity to build on knowledge, skills and creative activity generated by the Royal College of Physicians Educator Program.

7. Goal: To enable primary care faculty to engage in scholarship related to medical education.

8. Objectives: Master Educator Program fellows will: 1.Continue to build skills in teaching, curriculum development, learner assessment and educational program evaluation.  “Teaching” 2.Complete a scholarly medical education project that meets standards for academic presentation or publication.  “Scholarship” 3.Share knowledge and skills related to medical education by presenting this material to other medical school faculty.  “Dissemination”

9. Methods Subjects: Faculty were invited to apply to participate by submitting a one-page project proposal. Proposals screened for relevance and feasibility. First cohort: 5 fellows, Jan. 2009 to June 2010. Second cohort: 5 fellows, July 2010 to June 2011.

10. Educational Strategies and Resources: 1.Training: Biweekly 1 hour seminars for research skills, project discussion and peer feedback. 2.Mentoring: By experienced researchers in Primary Care Research Institute. 3.Pilot project funds: Up to $2,000 plus assistance in applying for external funding. 4.Project assistance: Staff support to reduce faculty time required for various research tasks.  No stipend, no time released from other duties.

11. Seminar Topics: Fall Introduction Linkage with mentors Project methodology Human subject research and institutional review board protocols Literature search and review Managing references Curriculum development Project discussions Spring Project implementation and evaluation Qualitative research Presenting results: posters, oral presentations Examples of project to publication Writing and publishing Teaching portfolio Advancing in academia Project discussions

12. Evaluation: 1.De-identified, IRB-approved program evaluation survey used to ascertain fellows’: Satisfaction with program. New skills for teaching and scholarship. Project characteristics. Presentations to other faculty. Suggestions for improving the fellowship.

13. Evaluation, cont’d: 2.Supplemental questionnaire used to ascertain external recognition of fellows’ accomplishments related to program objectives: Teaching awards Presentations Publications

14. Results Program evaluation survey: Completed by 3 fellows in first cohort May 2010. Completed by 5 fellows in second cohort May 2011. Supplemental questionnaire: Project data and outcomes obtained for all fellows in first and second cohort.

15. Satisfaction with program content and resources: Training seminars very good or outstanding. Project presentations and peer feedback very useful. Mentoring, pilot project funds, project assistance very useful. Agreement or strong agreement with statement, “I would recommend this program to a colleague.”

16. Development of skills and outcomes related to program objectives: Objective 1 – Teaching: Improved teaching skills. 3 fellows received teaching awards.

17. Objective 2 – Scholarship: Demonstrated and somewhat or very confident in the following skills: Project proposal Literature review IRB protocol Plan methods Implement project Analyze data Present results Prepare manuscript

18. Objective 2 – Scholarship, cont’d: Projects built upon Education Initiatives Project categories: Curriculum development. Learner assessment. Feedback. New assessment instruments. Care of patients in underserved or minority populations. ACGME core competencies.

19. Objective 2 – Scholarship, cont’d: External recognition : 10 academic presentations; (2 local, 8 regional or national). 1 research award. 1 published manuscript. 4 manuscripts in progress.

20. Objective 3 – Dissemination: Presentations: Own teaching site. Multiple teaching sites in own department. Teaching sites in multiple departments. Medical-school wide.

21. Suggestions for improving the fellowship: Protected time. More training in data analysis. Schedule for expected completion of steps in project work.

22. Conclusions Summary of findings: Results suggest that the Master Educator Program has been effective in enabling primary care faculty to engage in scholarship related to medical education based on evaluation at Kirkpatrick’s: Level 1 (learners’ reactions). Level 2b (acquisition of knowledge and skills). Level 3 (change in behavior).

23. Strengths: Suitable for clinical faculty who want to: Continue to build skills in teaching, curriculum development, assessment and educational program evaluation. Begin to develop their educational innovations into scholarship that meets standards for academic presentation or publication. Share knowledge and skills for medical education with other faculty.

24. Strengths, cont’d: External recognition of scholarship related to medical education may increase opportunity for reward via promotion and tenure. Program has attracted participants from specialties outside of primary care per se and fosters interdisciplinary exchange of ideas. Program advances the larger goal of building a “community of educators” within the medical school.

25. Limitations: Program evaluation was limited by: Incomplete response to program evaluation survey. (80%) Lack of evaluation at Kirkpatrick’s Level 4 (benefits to students or patients). Lack of a faculty control group. Program lacks depth in research training compared with programs that lead to graduate degree in education.

26. Future directions: Faculty development program to enhance the primary care workforce. Sharing of educational strategies with other medical school faculty.

27. Acknowledgements University at Buffalo Offices of the Dean, Graduate Medical Education and Continuing Medical Education Royal College of Physicians Department of Family Medicine Primary Care Research Institute U.S. Department of Health and Human Services, Grant no. D55HP05163

28. Fellows’ projects – First Cohort:  Using patients’ evaluations to improve residents’ communication skills and professionalism. (GIM)  Improving residents’ conference presentation skills using a brief assessment tool for peer feedback. (GIM)  Evaluating and enhancing a women’s health curriculum in an internal medicine residency program. (GIM)  Improving evidence-based journal club for family medicine residents. (FM)  Curriculum for teaching medical students to care for patients with developmental disabilities. (FM)  Developing a Web-based, anonymous system for residents to report patient- safety concerns. (FM) Fellows’ projects – Second Cohort:  Team skills training for medical students in 3 rd year surgery clerkship. (Gen. Surg.)  Curriculum for medical students on social and ecological determinants of health and exposure to certified community health workers. (Med./Ped.)  Comparison of spiritual assessment scales for primary care and rural-practice residency curriculum. (FM)  Evaluation of a medical student curriculum on spirituality in medicine. (FM)  Curriculum for residents to assist patients with acquired disabilities in obtaining financial benefits by improving documentation. (OB/FM)