1. © Medical College of Georgia 2007 Recognizing, Evaluating, and Rewarding Educators and Educational Scholarship Health Professions Educational Research Symposium January 14, 2007 Fort Lauderdale, FL Ruth-Marie E. Fincher, MD Vice Dean for Academic Affairs Medical College of Georgia School of Medicine

2. “Education Is Our Core Mission” Research Service Patient Care Education From Michael Wilkes, 2003

3. How It Seems Research Service Patient Care Education Modified by R Fincher, 2004

4. Making Us Feel Like...

5. Agenda for Presentation Background –Teaching & educational scholarship Framework –Assessing teaching & learning GEA scholarship project –5 categories of educational activities –Assessment of each category Quantity Quality Engagement with educational community –Scholarly approach –Scholarship

6. Changing Institutional Culture May Be More Difficult than Moving a Graveyard The Scholarship Culture is Changing Swanson G. In Tosteson et al. New Pathways to Medical Education. Cambridge MA, Harvard University Press, 1994 Slide adapted from Malcolm Cox

7. Teaching and Its Scholarship Are Often... Overshadowed by –Research –Clinical care Undervalued in reward system –Promotion –Tenure –Salary

8. Institutions Reward What They Value Teaching is highlighted in most mission statements Teaching and related scholarship must be rewarded if they are truly important to an institution

9. Recent Evolution of Medical School Environment Caused a “Crisis of Mission” ResearchTeachingClinical Care FundingSource (Is it?) Cost (Is it?) Source KnowledgeDiscoveryTransmissionApplication Value“Opportunity”“Obligation”Critical for survival Concept from Cuban: How Scholars Trumped Teachers, 2001

10. Think outside the box!

11. Medical Schools Only Unique Mission = Teaching....“widespread agreement that those members of the faculty who are most committed to... the education of medical students must be supported and rewarded.... Faculty appointments, promotion and tenure policies must reflect the changing roles and responsibilities of medical school faculty.” Whitcomb M: Acad Med 78:117-118, 2003.

12. Evolution of Concept of Teaching and Scholarship Pre-Boyer –Teaching = academic citizenship Everyone does it Little to do with scholarship or promotion Boyer –Reframed discussion by Challenging concept that “everyone teaches” Examining teaching as a form of scholarly work Boyer: Scholarship Reconsidered, 1990.

13. Continued Evolution: Glassick* Common criteria form foundation for all forms of scholarship Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique * Scholarship Assessed, 1997.

14. AAMC’s Group on Educational Affairs: Promotes Education Across Continuum of Medical Education Undergraduate Continuing Graduate RIME

15. GEA Scholarship Project Phase 1 1996-2000 Phase 2 2001-2004 Phase 3 2005-2006 Phase 4 2007- ? GOAL: Value faculty who support and advance medical education by connecting education, scholarship and academic advancement

16. GEA Scholarship Project Phase 1 1996-2000 Phase 2 2001-2004 Phase 3 2005-2006 Provided examples of activities and evidence Simpson & Fincher (1999) Defined educational scholarship Outlined infrastructure needed to support educational scholarship Fincher et al. (2000) Phase 4 2007- ?

17. Characteristics of Scholarship Demonstrates the “3 P’s”: –Product that can be reproduced and built upon –Open to peer review –Publicly disseminated Scholarship moves field forward Assessment parallels research

18. Phase 2 2001-2004 GEA Scholarship Project Phase 1 1996-2000 Phase 3 2005-06 Identified types of educational activities & evidence Simpson et al. (2004) Characterized infrastructure – Academies Irby et al. (2004) Phase 4 2007- ?

19. Education Activity Categories Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:783-790

20. Infrastructure to Support Education Faculty development centers –Career development –Teaching/educational research skills Societies or Academies –Recognize accomplishments based on selection criteria

21. Phase 2 2001-2004 GEA Scholarship Project Phase 1 1996-2000 Phase 3 2005-06 GEA Consensus Conf on Educational Scholarship Defined areas of agreement & concerns in 3 areas Category inclusions Evidence & presentation displays for each category Areas needing further investigation Phase 4 2007- ?

22. Categories of Education Activity Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:783-790

23. Q 2 + Engagement with Community Contribution to education mission Quantity Quality Engagement with education community Scholarly approach to education activity Educational scholarship Draws FromContributes To

24. Scholarly and Scholarship Existing literature Best practices Resources in field Colleagues locally  internationally Dissemination –Peer reviewed forums Local  international audiences Impacts the field Scholarly work is influenced by: Scholarship contributes to the field:

25. Documentation: Teaching Category Brief Description –Role, activity description Evidence of Quantity –Who, what, when, where, how much, how many Evidence of Quality –Summary of student/peer evaluations –Short excerpts from supporting letters –Invitations to teach outside department or school –Repeat invitations to teach same group or in course

26. Teaching Definition and Inclusions Is the design and implementation of activities to promote learning Includes –Course design –Development of instructional materials –Interactions with students –Formative and summative assessment Act of teaching is not scholarship

27. Teaching: Quantity & Quality RoleYr(s)QuantityNumber Learners Quality Family Medicine Clerkship Seminar Leader 1997-~19 hr/ year 8-12/ rotation; 8 rotations/ year ’05-06: “Effective seminar leader” Mean = 5.41 (7-point scale)* Give comparative ratings for each year; compare with *Comparative ratings should be given over time; Compare with peer group if possible *Give comparative ratings for each year; provide comparative ratings with peers if possible

28. Teaching: Quantity and Quality

29. Engagement with Community Scholarly approach: Learns from –Educational literature –Best practices Scholarship: Contributes to field –Product: Interactive cases –Peer review: MedEdPORTAL –Public: Web

30. Scholarly Approach Teaching - learning connection –Consult education literature –Apply intervention to enhance learning –Observe outcomes –Analyze results –Obtain peer evaluation –Use results to improve teaching Product reflected in student learning Richlin L: Scholarship Revisited: Perspectives on the Scholarship of Teaching. Jossey-Bass, 2001

31. Evidence of Scholarship Product: Instructional material –Interactive, diagnostic decision making cases: Cough, chest pain Describe role Peer review and public dissemination –Website: xx hits; xx schools adapted or adopted at least 1 case; representative feedback comments –Accepted by national peer-reviewed venue; e.g., MedEdPORTAL

32. Curriculum Development Definition and Inclusions Longitudinal designed educational activities –Any training level –Various venues and delivery formats Includes –Goals and objectives –Learning experiences to achieve goals & objectives –Organization and sequencing to ensure effective learning –Evaluation of effectiveness

33. Curriculum Development Activity and Role Title of course –Evidence-based Medicine (Yr. 1 students) Role: Course director –Organized all lectures and self-directed learning exercises for 5 credit hour course Developed objectives and content Recruited and trained facilitators Worked with clinicians and statisticians to develop illustrative cases

34. Engagement with Community* Clear Goals –Create a new EBM course for all first-year students that students perceive as clinically relevant. The predecessor course consistently received “very poor student evaluations” and EBM content was “lacking in the curriculum.” Adequate Preparation –PhD in Public Health –Review of “best practices: McMaster’s curriculum, NBME test content –Review of literature; application to new course *Applying Glassick’s Criteria

35. Engagement: Glassick  Appropriate Methods  Multi-method approach including:  Interactive lecture series  Clinical vignettes  Abstract followed by article critique  Increased collaboration  Small groups co-led by basic scientist/MD  Assessment methods: Test questions assessed application of knowledge

36. Engagement  Significant Results (Outcomes)  Didactic series evaluation  Improved 1 SD from prior offering  AAMC Graduation Questionnaire  EBM rating improved from inadequate exposure to appropriate/excessive  USMLE sub-test score  Improved from below to above average  OSCE performance on ambulatory practice module  Improved

37. Engagement with Community: Scholarship (Contributing to) Effective Presentation (Dissemination) –Results presented to curriculum committee –Internal review; comparison over last 2 years –Content replicated in clerkships & residencies Reflective Critique (Next steps for continuing improvement) –Convert course to on-line format to minimize # faculty needed –Submit to AAMC MedEdPORTAL (Peer review)

38. Advising and Mentoring Definition and Inclusions Advising and mentoring –Educator provides guidance or counsel to facilitate accomplishment of a learner’s or colleague’s goals Advising –Limited time; advisor serves as guide to enable advisee to achieve her or his goals Mentoring –Sustained relationship; mentor and protégée obtain reciprocal benefits

39. Advising & Mentoring NameCW: Medical Student  Medicine Resident Duration/ Process 9/01 - present: 4 mtg/yr; Edit (paper, CV, application to be LCME student liaison) OutcomeMD Received; Paper published in Acad Med; student liaison to LCME Effective- ness “I wanted to thank you for … making me the young man I am today…not sure you realize impact you’ve had in my life…” Engage- ment Scholarly: Learned from literature, professional development activities, etc. Scholarship: Mentor manual adopted by others; workshop at national meeting

40. Educational Leadership Definition and Inclusions Definition –Exceptional leadership transforms educational programs and advances field Inclusion criteria –Pursues excellence –Evaluates and engages in self-reflection –Builds on work of others –Disseminates results – advances the field –Garners and maximizes resources

41. Leadership Documentation Quantity –Describes nature of activity, duration, rationale for change, goals, and leader’s role Quality –Describes actions, accomplishments, evaluation and resources garnered/utilized Engagement with educational community –Builds on literature and best practices –Disseminates and reflectively critiques

42. Leadership Project Basic Science/Clinical Integration Need –Courses lacked clinical relevance, poorly taught Goal –Increase integration across 4 years, clinical relevance, quality Methods and preparation –Revised courses –Conducted faculty development during 2004-06 –Garnered resources: Support staff and faculty time Results (Evaluation) Presentation (Peer review and dissemination –6 peer reviewed activities Reflective Critique –Next year plan to… Rating20032005 Overall3.44.3 1 = Poor 5 = Outstanding

43. Learner Assessment Definition and Inclusions Activities associated with measuring learners’ knowledge, skills, and attitudes Assessment-related activity(ies) include: –Development Identifying and creating assessment processes and tools –Implementation Collecting data using processes and tools –Analysis Comparing data to correct answer key and/or performance standards –Synthesis and presentation Interpreting and reporting results

44. Learner Assessment Documentation Quantity –Type of assessment –Learner population E.g., who and how many –Size/scope E.g., number of items, frequency of use –Intended uses of results –Faculty member’s role Quality and engagement –Evidence of adherence to Glassick’s criteria

45. Learner Assessment Example of Presentation

46. Results: Caveats & Remaining ?s Each institution must determine what counts for promotion –Contributions to institution’s mission (Q 2) Q 2 within and across categories Sustained contributions? –Contributions to the field (Engagement) Draw from and contribute to field Sustained contributions? –How to value individual vs group contributions Core elements education infrastructure

47. GEA Scholarship Project Next Steps Phase 1 1996-2000 Phase 2 2001-2004 Phase 3 2005-06 Dissemination & Infrastructure: Individual & Institutional to Support Educators AAMC-GEA Annual & Regional Meetings + Publish Institutional Team Workshops Individual Faculty Workshops Phase 4 2007- ?