Developing Learning Assessment Plans and Outcomes Assessments: Challenges and Opportunities in Medical Education Research Robert A. DiTomasso, Ph.D., ABPP Professor & Chairman Department of Psychology Director of Institutional Outcomes Assessment PCOM

Three Critical Concepts Learning-a relatively permanent change in behavior that occurs as a result of the experience of a student and is not due to other causes Assessment-a formalized, systematic process of determining the extent to which learning has occurred Outcome-reliable and valid evidence that learning has occurred

Use of Institutional Research Evidence of accomplishing mission Continuing self-assessment process Connect strategic plan to learning outcomes assessment Assessing student achievement, program effectiveness, and opportunities for improvement Measure clinical competencies

Accreditation Standards Outcomes=the results of the COM’s efforts in attaining its missions and goals

Learning Assessment Plan An accurate, practical, comprehensive, detailed, efficient, effective, valid, reliable, assessment plan or scheme specifically designed to evaluate the nature and extent to which student learning is being achieved

Learning Assessment Plan Delineates the relationship between institutional, programmatic, and course level goals and the expected student learning outcomes Documents the congruence between the aims of an educational organization and its administration, programs, faculty, students, and other consumers

Learning Assessment Plan Identifies –learners –teachers –sources of learning –types of learning experiences –nature-timing-sequence-organization of these learning experiences –formative and summative learning outcome assessments

Learning Assessment Plan Unique plan for the DO program Addresses what is absolutely most important to know about outcomes Easy to implement and manage Easy to understand and explain Non-threatening Well organized and methodical

Learning Assessment Plan Time sensitive Feedback-based Utilizes multiple assessments, modalities, informants, time points Relies on institutional commitment

10 Important Reasons For Developing A Learning Assessment Plan 1. Failing to plan is planning to fail 2. It’s better to know than assume 3. Absence of evidence is evidence of absence in this case 4. If you don’t ask the important questions, someone else surely will 5. The best defense is a good offense

10 Important Reasons 6. Only the strong survive 7. Some questions may never be truly answered without resorting to data 8. To ask is to know 9. There is strength in numbers 10. Information is power

Functions Served By LAPs Linking Informational Directive Pinpointing Reinforcement Mapping

Advantages of LAPs Relates numerous processes to multiple critical outcomes Thread that connects or binds diverse educational goals and activities Guides the educational improvement process

Advantages of LAPs Relates –what needs to be learned, –how it is to be learned –how we know it’s learned/not learned –what we can do to make it better

Disadvantages of LAPs You may find out what you don’t want to know (but need to know) May demolish some cherished beliefs or practices

Important Considerations For Success Outcomes assessment must become part of the culture of the institution Outcomes assessment is not done for accreditors and then forgotten about until the next review Outcomes assessment is a critical pathway along the road to excellence

Important Considerations For Success Outcomes Assessment is necessary and sufficient for –documenting educational quality –maintaining educational quality –improving educational quality

Components of LAPs 1. Delineation of core competencies students are expected to acquire 2. Identification of all course and course related educational experiences directly related to each competency 3. Identification of all formative and summative assessment measures

Components of LAPs 4. Identification of the point in program when assessment is conducted 5. Specification of exact outcome information obtained 6. Specification of feedback loops

10 Step Process 1. Obtain institutional support/ Involve key players in process 2. Delineate the specific core competencies you expect your students to acquire 3. Identify all courses and related educational experiences directly related to each competency

10 Step Process 4. Identify the all formative and summative assessment measures

10 Step Process 5. Identify point in program when assessment is conducted 6. Ascertain that data to be obtained will reliably and validly measure the competency 7. Assume a collaborative focus with an eye toward excellence

10 Step Process 8. Design a system for data collection that is efficient, minimally intrusive, and embedded within the program 9. Provide assistance with data entry, analysis, interpretation, and reporting 10. Develop a system for feeding information back to each program, faculty, administration, committees, and students

THE END