PORTFOLIO ASSESSMENT (Undergraduate Medical Education) Leilanie A. Nicodemus, M.D., MsCM-FM

OBJECTIVES Discuss the principles of portfolio assessment in relation to Outcome-Based Education Discuss rubric as method of assessment of student portfolios. Discuss the Dundee Case Study of Portfolio in undergraduate medical education for OBE. Reflective Portfolio of UPCM in 2005

THE RHETORIC

ASSESSMENT DEFINED A systematic data gathering process regarding student learning, using appropriate methods, standards and criteria, in order to collect information that will serve a number of purposes. Informing decision-making about student ability Informing the decision about whether the student has passed the course Providing data for feedback to students Contributing elements to the evaluation of the effectiveness of the educational process. Norcini,J. and Troncon,L. (2015) Foundations of assessment. FAIMER-Keele Master’s in Health Professions Education: Accreditation and Assessment. Module 1, Unit 1

MILLER’S PYRAMID OF ASSESSMENTS

Portfolio assessment for OBE Authentic Assessment Portfolio assessment for OBE It facilitates assessment of integrated and complex abilities and takes account of the LEVEL and CONTEXT of learning. AMEE Medical Education Guide No 24 Friedman Ben-David M, et al (2001)

PORTFOLIO ASSESSMENT

PORTFOLIO’S CONTRIBUTION TO ASSESSMENT The assessment of learning outcomes The provision of evidence of performance The representation of evidence collected over a period of time Student progression towards the learning outcomes Summative and formative assessment

STRENGTHS OF PORTFOLIO ASSESSMENT IN-DEPTH PROFILE OF LEARNERS ABILITIES Summative evaluation statements may also contain information regarding students’ strengths and weaknesses, thus integrating summative decisions with formative profiles.

PSYCHOMETRIC ISSUES to be addressed during PLANNING Portfolio’s main assessment features Formative and summative Qualitative and quantitative Personalized Standardized Authentic Rater’s consistency Decision consistency Sampling – breadth and depth (BEST WORK) The consensus approach Forms of validitiy – predictive validity External vs internal examiners

RUBRICS Is a scoring guide or specific pre- established performance criteria in which each level of performance is described to contrast it with the performance at other levels

Rubrics in Portfolio Assessment To assess multiple outcomes simultaneously or for multidimensional outcomes and each dimension needs to be rated separately, resulting in multiple judgments about an object or performance.

Steps in WRITING RUBRIC Set the Scale Define the Ratings Identify the basic descriptions Descriptions of what will look like in each level

WRITING RUBRIC: Set the Scale Select a learning outcome Use your professional judgment to ASSESS learner’s learning on a SCALE 1-4

WRITING RUBRIC: Set the Ratings Add appropriate descriptors to each number on the scale that you have identified like: 1: Beginning 3: Competent 2: Developing 4: Accomplished 1 2 3 4

WRITING RUBRIC: Identify Basic Descriptions 4 – Accomplished is __________ 3 – Competent is ___________ 2 – Developing is _____________ 1 – Beginning is ________________. 1 2 3 4

WRITING RUBRIC: Descriptions of what performance will look like at each level 4 - The student is able to (description of what ACCOMPLISHED would look like). 3 – The student is able to (description of what COMPETENT would look like) but not yet able to (description of ACCOMPLISHED performance). 2 - The student is able to (description of what DEVELOPING would look like) but not yet able to (description of COMPETENT performance). 1 - The student is BEGINNING to (description of desired performance).

EXAMPLE 1 2 3 4

THE ACTION

University of Dundee School of Medicine Case Study (2001): Portfolio Assessment

INNER CIRCLE: “WHAT THE DOCTOR IS ABLE TO DO” Clinical Skills Practical Procedures Investigating a patient Patient management Health Promotion and disease prevention Communication skills Information Handling

MIDDLE CIRCLE: “HOW THE DOCTOR APPROACHES THE TASK” Understanding of basic, clinical and social sciences, Appropriate attitudes, ethical understanding and legal responsibilities Appropriate decision making, clinical reasoning and judgment.

OUTER CIRCLE: “DOCTOR AS A PROFESSIONAL” The role of the doctor within the health service Aptitude for personal development.

University of Dundee School of Medicine Case Study (2001) Goal: Assess the achievement the outcomes at the appropriate level expected of a graduating student of the OBE curriculum. Key Feature: As key component in the FINAL UNDERGRADUATE DEGREE EXAMINATION, in order to assess aspects of medical practice such as professional development.

Mechanics: Dundee Case Study Step 1

Step 2

Step 3

Step 4 MINIMUM PASS – no final examination DISTINCTION – examination to get ranking of the students REFERRED – remedial (diagnostic examination and remedial rotation)

Reflective Portfolio UPCM 2005 for Integrated Clinical Clerkship (LU 5)

REFLECTIVE PORTFOLIO UPCM for the Integrated Clinical Clerkship (Ambulatory) Goal: Promote reflective learning of Clinical Clerks in Family Medicine Components: Self-assessment – strengths and weakness in handling cases in primary care Personal objectives – compare to the objectives of the rotation Identification of resources (inter and intra) Collection of significant learning events Sign-out reflection regarding the rotation Processing of Learning Portfolio: 2 faculty members do portfolio interview (focusing on “growth attained” during the rotation) FORMATIVE Completeness and accuracy of data of the learning portfolio SUMMATIVE

Student Feedback “Helped me think of WHAT I should learn” (content) “It made me make sense of the rotation” (self-directed learning) “Helped me consider primary care as practice” (future career) “It is time consuming to do especially that we have other papers to submit” (synchronization of learning activities) “We are given 5 points in the total grade which may not be fair with the efforts put into the portfolio.” (summative vs formative )

Faculty Feedback “Makes me realize the potential of each student” (mentoring) “Makes me more active in teaching students especially correct misperceptions or wrong learnings during the rotation.” (coaching) “Time consuming but worth it.” (clinical teaching) “I need to learn counseling.” (clinical teaching/preceptorial)

Department “We are popular to the students.” (relevance) “You (pertains to the coordinator) need to find a space for storage.” (logistics) “There was a complaint from the portfolio submitted to the College, you need to answer that.” (accountability) “Can you (pertains to the coordinator) increase the grade for the portfolio.” (summative assessment)

In Summary

REMEMBER! Portfolio in not just a collection of learner’s work but it is a collection of his BEST WORK! Developing a portfolio is arduous for the learners that should be valued. Assessment is a way to give meaning to the collection of best works and will give feeling of fulfillment to the learner of a JOB WELL DONE! LEILANIE A. NICODEMUS, M.D. 31st FAMED Annual Convention Ilocos Norte