Curriculum Inventory Administrators’ Group February 10, 2016 Terri Cameron, MA, Director, Curriculum Programs

Please use the Chat panel located on the right hand side of your screen to submit your questions. Send to All Panelists. Questions

CI Upload Statistics January CI in Context: Academic Levels in US and Canadian Medical Schools February CI in Context: Use of Resources in US and Canadian Medical Schools Updated CI Standardized Vocabulary Featured Report: Regional Medical Campuses MedBiq CI Standard Review: Multi-site Clerkship Options Curriculum Inventory Report Template (Now active) AAMC Medical Education Cluster Update CI at 2016 Medical Education Meetings Medical School Highlight: University of Nevada School of Medicine Brady Janes, Director, Curriculum Development and Assessment, and Gwen Shonkwiler, Director, Evaluation and Assessment Next meeting: Wednesday, March 9, 1 pm ET Agenda

CI in Context: January Academic Levels in US and Canadian Medical Schools N. Kevin Krane, MD, Tulane University School of Medicine

CI in Context: February Use of Resources in US and Canadian Medical Schools Kristi Ferguson, MD, Director, OCRME University

Curriculum Inventory Standardized Vocabulary Update New: Exam – Institutionally Developed, Laboratory Practical Remove

Curriculum Inventory Standardized Vocabulary Update Independent Learning: Instructor-/ or mentor-guided learning activities to be performed by the learner outside of formal educational settings (classroom, lab, clinic) (Bowen & Smith, 2010); Dedicated time on learner schedules to prepare for specific learning activities, e.g., case discussions, TBL, PBL, clinical activities, research project(s). Synonymous with/Includes: Independent Study; Self-Study, Homework Does not include: Self-Directed Learning Self-Directed Learning: Learners taking the initiative for their own learning: diagnosing needs, formulating goals, identifying resources, implementing appropriate activities, and evaluating outcomes (Garrison, 1997; Spencer & Jordan, 1999). Does not include: Independent Learning; Independent Study Only time that is blocked out on the calendar should be coded as self-directed learning or independent study

Curriculum Inventory Standardized Vocabulary Update Direct Observation (with thanks to Hugh Stoddard, Emory): two reliable sources point towards our interpretation that using a “checklist or rating” is a method, not a resource (and that “observation” is not a method of assessment). In the first chapter of the ACE Handbook, Bill McGaghie includes both ‘checklist’ and ‘rating scale’ in Table 1 of Chapter 1 which shows the “Evaluation Methods Commonly Used in Medical Education.” He did not include “observation” as a method anywhere in this table of 16 commonly used methods. (Pangaro, Louis N., and William C. McGaghie, eds. Handbook on Medical Student Evaluation and Assessment. Gegensatz Press, 2015.)

Curriculum Inventory Standardized Vocabulary Update More importantly, at the theoretical level, the quotes below from chapter 1 of a book sponsored by the National Research Council explain the necessary characteristics of an assessment (Pellegrino, James W., Naomi Chudowsky, and Robert Glaser. Knowing what students know: The science and design of educational assessment. Washington (2001). [ “Every educational assessment, whether used in the classroom or largescale context, is based on a set of scientific principles and philosophical assumptions” “First, every assessment is grounded in a conception or theory about how people learn, what they know, and how knowledge and understanding progress over time. Second, each assessment embodies certain assumptions about which kinds of observations, or tasks, are most likely to elicit demonstrations of important knowledge and skills from students. Third, every assessment is premised on certain assumptions about how best to interpret the evidence from the observations to draw meaningful inferences about what students know and can do.”

Curriculum Inventory Standardized Vocabulary Update The assessment methods of using a checklist or rating scale are based on assumptions about what is known, how that knowledge is manifested, and how results should be interpreted. (Granted, many Med Ed programs simply use checklists/ratings as tools and do not give much attention to the underlying theory of cognition or interpretation.) On the other hand, “observation” is a means of gathering data that contributes to many assessment methods, but we have not found literature to support learning theory or psychometric principle that would be inherent in the act of ‘direct observation’ itself.

Featured CI Report: Regional Campuses

MedBiq CI Standard Review: Multi-Site Clerkship Options

Curriculum Inventory Report Request Template Drop-down will include ONLY participating schools Will be adding options to include data from: All Schools US MD CA MD US DO

Reports Requested in January: Clinical Diagnosis (“Diagnosis”, “Introduction to Clinical Medicine”, “ICM”, and “Doctoring) Basic Science Disciplines (29) Sports Medicine Musculoskeletal Orthopaedics Rheumatology Physical Medicine Gun Safety Opiods Pain Management Adverse Childhood Experience Eating Disorders Nutrition

AAMC Medical Education Cluster Update Lisa Howley, M.Ed., Ph.D. -- new Senior Director of Educational Affairs The deadline for applying for the vacant Medical Education Chief position was January 25. Robby Reynolds, M.P.A. -- leaving the AAMC. Walter and Terri will continue to lead the MedAPS Initiative.

Medical Education Meeting and Learn Serve Lead The AAMC Annual Meeting and the AAMC Medical Education Meeting will be reunited in The meeting will be held in Seattle November

CI at 2016 Medical Education Meetings AACOM (April, Washington, DC) CCME (April, Montreal) GEA Regional Meetings: CGEA (April, Ann Arbor) NEGEA (April, Newport) SGEA (April, Austin) WGEA (April, Tucson) MedBiquitous Conference (May, Baltimore) GIR (June, Toronto) IAMSE (June, Leiden) AMEE (September, Barcelona) ? Learn Serve Lead (November, Seattle)

CI Upload Portal opens August 1; closes September 30. Staging open now, but not yet converted to Conversion originally planned for April; will try to move to March. Goals: US medical schools: 100 percent participation (148) Canadian medical schools: 50 percent participation (9) Osteopathic School Pilot: 30 percent participation (10)

Curriculum Reform and Renaissance Through Assessment Brady Janes, MS, MA Director, Curriculum Development and Assessment Gwen Shonkwiler, PhD Director, Evaluation and Assessment

Past, Present, On-Going, Future 22 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus

23 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Curriculum Reform Fall 2012: Year 1 & 2 Implementation Departmental  Integrated Basic Sciences + Systems-Based + Clinical Context In process: Year 3 & 4 Review and Reform Addition of Clerkship in Neurology Addition of Selectives Addition of a Capstone Course

24 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Trend Analysis Year 3 – 4 Curriculum Reform Clerkship Disciplines Average Clerkship Length by Discipline Longitudinal Clerkship Design Elective Rotations at Other Institutions

25 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Alignment Institutional Objectives Medical Knowledge, Patient Care, Practice-Based Learning, Interpersonal Communication Skills, Professionalism, Systems-Based Practice ~40 Topics and Content Areas ie: Pain Management, Genetics, Anatomy, Nutrition, etc. Block Objectives Weekly Objectives Activity Objectives

26 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Mapping, Gaps & Redundancies In-depth Analysis of Curriculum Years 1 -2 Identifying gaps – what instruction do our students need in order to be successful in future Block(s), Exams/Assessments, Patient Encounters, Professional Goals, etc… Identifying redundancies – what did we cover that should be covered again, deeper, or not repeated? Focus Groups Curriculum Management System Clerkship Preparedness Survey

27 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Assessment What do we want our students to be able to do, understand, perform, apply upon completion of the Block, Clerkship, Elective, Activity, Lecture, Encounter, Simulation, etc…? How do we assess outcomes? Core Entrustable Professional Activities Physician Competencies Reference Set

28 Continuous Quality Improvement Curriculum Reform Trend Analysis Alignment Mapping, Gaps & Redundancies Assessment Direction & Focus Season for Strategic Planning Addition of UNLV SOM LCME Accreditation Review Opportunity to Focus on Identity, Distinction, and Intentional Student Outcomes

Curriculum Reform and Renaissance Through Assessment Brady Janes, MS, MA Director, Curriculum Development and Assessment Gwen Shonkwiler, PhD Director, Evaluation and Assessment

Wednesday, March 9, 1 pm ET (Second Wednesday of each month, 1 pm ET) Registration Links posted in Training and Resources section of Please send agenda items to Next meeting: