1. FAIMER Assessing Teaching Excellence John Norcini, Ph.D.

2. Overview Purposes Competencies Methods Challenges Good system of teacher evaluation Supporting teaching excellence Education: That which discloses to the wise and disguises from the foolish their lack of understanding Ambrose Bierce FAIMER

3. Purposes Assess individual teachers  Help faculty improve  Establish accountability and make decisions Assess educational programs  Improve curriculum and courses  Generate reform “Good teaching is one- fourth preparation and three-fourths pure theatre.” Gail Godwin FAIMER

4. Competencies Competency framework is essential but often missing (Hardin)  Competencies Information provider, Role model, Facilitator, Assessor, Planner, Learning resource developer  Approach Scientific principles, Attitudes and ethics, Decision making, Teamwork  Professional development FAIMER

5. Methods Teaching activity Student ratings Peer ratings External expert ratings Self-evaluation Video Student interviews Alumni ratings Employer ratings Administrator ratings Teaching scholarship Teaching awards Learning outcomes Teaching portfolio FAIMER Berk, 2005

6. FAIMER Student Ratings Students rate teaching skills  Most popular measure of attributes like clarity, interpersonal rapport, concern for students...  Useful for formative, summative, and program evaluation  Thousands of studies with good evidence of reliability and validity Challenges include the quality of the rating forms, assortment of biases...

7. FAIMER Learning Outcomes Using student performance to judge teaching skill  Assessment of outcomes rather than process  Establishes accountability  Best for formative and program assessment Challenges include whether there are differences in ability of students to start and whether there are influences on students beyond the teachers...

8. FAIMER Teaching Portfolio Systematic collection of a variety of measures  Includes teaching effort, ratings, advising/mentoring, educational leadership- administration, learner assessment...  Judged by quantity, quality, scholarship  Good for both formative and summative assessment Challenges include comparability, scoring standards, assessors...

9. Methods Substantial number of methods  All add something  All have limitations Important to rely on a system of assessment rather than individual measures  Incorporate several sources of information  Capitalizes on the strengths of several methods and balances out deficiencies FAIMER

10. Challenges in Teacher Assessment System of assessment  Nonexistent  Unfocused For example, it relies on general satisfaction and does not include student achievement…  Undifferentiated Satisfactory versus unsatisfactory  Unhelpful Feedback does not inform improvement  Inconsequential FAIMER

11. Challenges in Teacher Assessment Faculty are threatened Faculty do not believe it is needed No culture of teaching Not enough faculty development Feasibility Community-based preceptors FAIMER

12. Characteristics of a Good System of Teacher Assessment Annual process Clear rigorous expectations Multiple measures Multiple ratings Regular feedback Significance

13. Supporting Teaching Excellence Assessment of teaching excellence is most effective in the context of a continuous quality improvement program  Faculty development  Provides appropriate recognition FAIMER

14. Faculty Development Faculty are trained to be health care providers but not teachers  Faculty development programs are intended to prepare faculty for their various roles  Help address faculty shortages  Empower faculty members to excel as educators  Create a community that values teaching and learning

15. Faculty Development Effectiveness evidence (Steinert et al, 2006)‏  Faculty development participants found the programs acceptable, useful, and relevant  Tests showed significant knowledge gains  Changes in teaching behavior were reported by participants and detected by students  Changes included greater educational involvement and establishment of networks FAIMER

16. Recognition Significance of teaching assessment requires some form of recognition  Faculty are often not rewarded for teaching and educational activities Many cannot afford to devote much time to clinical education  Develop a system that recognizes, rewards, and promotes faculty, in part, based on their teaching activities

17. Recognition Recognition can take a variety of different forms  Inclusion in the tenure and promotions process has been used successfully  Academy of Medical Educators in the UK Recognizes commitment to medical education Standards for membership that include active engagement, course work, CPD… Acts as a credential that exceeds the minimum standards of the GMC for clinical trainers

18. Summary Goal of measuring teaching excellence is to improve the quality of care  Formative, summative, programmatic  Variety of methods  System and competencies are needed It is important to support teaching excellence  Faculty development and recognition "Some cause happiness wherever they go; others, whenever they go." Oscar Wilde