1. The Educator’s Portfolio: Creation and Evaluation
David M. Irby, PhD
Kanade Shinkai, MD PhD
Patricia O’Sullivan, EdD
University of California, San Francisco Academy of Medical Educators

2. Preview Identify the major roles of educators
Determine documentation standards and evaluation criteria for each role
Examine the Academy of Medical Educator’s Portfolio and begin creating your own portfolio
Review UCSF promotion processes and how a portfolio relates to Advance

3. Agenda for the session Introduction/Preview – educator roles (20 min)
Presentation on evaluation constructs (30 min)
Review and evaluation of a portfolio
Small group work—Exercise 1, 3 Parts (25 min)
Report Out (10 min)
Small group work—Exercise 2 (20 min)
Break (15 minutes)
Creating your UCSF portfolio (45 min)
Presentation - Advancement and Promotion (60 minutes)

4. Question What are the core roles of an educator? Think, pair, share
Slides available at
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

5. Curriculum Development
Educator Roles
Teaching
Activities
Curriculum Development
Learner
Assessment
Mentoring
& Advising
Educ Leadership
& Administration
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

6. Rationale for Educator’s Portfolio
Documents achievements, reflections and plans
Makes work visible and therefore peer reviewable
Enables education to be scholarly
Public
Peer reviewed
Platform to build upon
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

7. Timeline of National Initiatives
Education as Scholarship
Boyer
Glassick
Fincher
Documentation standards for Educational Scholarship
Simpson
Criteria for Evaluating Educational Scholarship
Gusic
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

8. The Scholarship of Teaching
Scholarship includes
Discovery
Integration
Application
Dissemination/teaching
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

9. Glassick’s Criteria for Scholarship
Clear goals
Adequate preparation
(e.g., work is informed by the literature and best practices in the field)
Appropriate methods
Significant results
Effective presentation
Reflective critique
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

10. Documentation standards for Educational Scholarship
5 domains of educator activity
Documentation requires
Quantity
Quality
Scholarly teaching (builds on others)
Scholarship of teaching and learning (advances)
Simpson D, et al. Advancing Educators and Education: Defining the Components and Evidence of Educational Scholarship Available at: aamc.org
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

11. Excellence and Scholarship
Criteria
Teaching Excellence
Scholarly Teaching
Scholarship of Teaching
Quantity & Quality
Builds on Field
Public
Peer Review
Contributes to Field
Simpson D, et al. Advancing educators and education by defining the components and evidence associated with educational scholarship. Med Educ. 41:1002–1009, 2007.
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

12. Kirkpatrick’s Model to Judge Results
Kirkpatrick Level
Assessment Level
Criteria in toolbox 1
Satisfaction/reaction
Of participants/stakeholders 2
Learning
Measures of knowledge, skills, attitudes, and/or behaviors 3
Application
Desired performance demonstrated in other settings 4
Impact
On education programs and processes within and/or outside institution
Kirkpatrick DL and Kirkpatrick JD. Evaluating Training Programs: The Four Levels (3rd Ed); 2006.
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

13. AAMC Toolbox for Evaluating Educators
Glossary
Executive Summary
User’s Guide
Toolbox with Tables of Indicators for Evaluation
Appendices
Reference list
Gusic M, Amiel J, Baldwin C, Chandran L, Fincher R, Mavis B, et al. Using the AAMC Toolbox for Evaluating Educators: You be the Judge!. MedEdPORTAL; Available from:
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

14. Small group work: Exercise 1
Part A
Review the teaching activities section of the faculty member’s portfolio
Discussion
How would this colleague be evaluated at UCSF?
Report out
Key points from the discussion
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

15. Exercise 1-Continued Part B Discussion Report out
Use the criteria from the toolbox to evaluate the teaching activities in the portfolio
Discussion
How do the criteria help you show that this faculty member’s work should be valued?
Report out
Which elements of the tool were most helpful to you?
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

16. Exercise 1-Continued Part C
Review the narrative summary about the educator that was created using the criteria from the toolbox
Large group discussion—Comments/impressions
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

17. Exercise 2 Review curriculum development portfolio Discussion
Use the criteria from the toolbox to evaluate the portfolio entry
Discussion
Identify areas of strength and targets for improvement
Report out
Which elements of the tool were most helpful to you?

18. Curriculum Development
Educator Roles
Teaching
Activities
Curriculum Development
Learner
Assessment
Mentoring
& Advising
Educ Leadership
& Administration
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

19. Break

20. Elements of a UCSF Portfolio
Personal statement
Documentation in one or more areas
Direct Teaching
Program Design & Curricular Innovation
Advising & Mentoring
Educational Administration & Leadership
Educational Research
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

21. Direct Teaching

22. Preparing your Educator’s Portfolio
The Educator's Portfolio
Faculty Advancement - Navigating UCSF
Preparing your Educator’s Portfolio
Choose categories of educational activity
Direct teaching required for AME application plus one other area
Start collecting data early
Write your personal statement
Request a trained AME coach
This really needs a lot of work – please help
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement
Jessica Muller, PhD
Martin Bogetz, MD 22

23. Personal Statement for UCSF Academy
Addresses the applicant's educational philosophy, professional development in education, and intended contributions to the Academy and to the School of Medicine (2 pages maximum)
Write a personal statement
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement

24. Resources These Slides UCSF Academy of Medical Educators Portfolio
UCSF Academy of Medical Educators Portfolio
AAMC Toolbox
Gusic M, Amiel J, Baldwin C, Chandran L, Fincher R, Mavis B, et al. Using the AAMC Toolbox for Evaluating Educators: You be the Judge!. MedEdPORTAL; Available from:

25. Portfolio and Advancement
Educator Roles
Evaluation Criteria
Portfolio Review
Advancement