1. Mapping Medical Education Curricula
Robby Reynolds, AAMC
Cindi Hasit, Cooper Medical School of Rowan University
Tomo Ito, Oregon Health and Science University
Rebecca Keller, Albany Medical College
Lynn Romrell, Florida State University
Terri Cameron, AAMC

2. What we will cover:
How to engage faculty in mapping, thinking of educational outcomes
Use of NBME ‘GEMS’ Tool
Linking learning objectives to assessment
New School common vocabulary
Curriculum Renewal and Mapping
Curriculum Transformation – moving toward CBLA
Curriculum Mapping vs Student Performance
Curriculum Inventory Options for Mapping

3. A New School’s Perspective
Cindi Hasit, PhD, Cooper Medical School of Rowan University

4. Cooper Medical School of Rowan University
Cooper Medical School of Rowan University (CMSRU) is located in Camden, NJ with a mission to serve the underserved.
“Camden is our classroom: Camden is our home.”
CMSRU’s first class entered in August 2012.

5. Curricular Highlights
Vertical and horizontal integration
Interprofessional education
Early clinical exposure
Focus on active learning
Case-based learning is 50% of class time
Flipped classroom model for select lecture topics

6. Organ- or discipline-based curriculum
Year 1 - PHASE I/Foundation & Integration
Year 1 - PHASE I/Foundation & Integration 1 8 4 5 9
Summer Elective
Orientation
Clinician’s Clinic
Fundamentals W O
Lifestages ID
Heme-Onc
Skin-Musc-Skel
Basic or Clinical Research
Ambulatory Clerkship
Community Svc
Scholar’s Workshop
Population Health
Foundations of Medical Practice
Year 2 – PHASE I/Foundation & Integration 8 1 6 3 5 4
Cardiovasc-Pulmonary W O
GI-Endo
Selective
Renal-Uro
Women’s Health
ENT-Allergy
Neuro-Psych
Prep for Step 1
Intro Phase II
Ambulatory Clerkship
Scholar’s Workshop
Foundations of Medical Practice
IPE: currently USP pharmacy students; will include nursing, social work, psychology, possibly PA and possibly law students
Organ- or discipline-based curriculum
Longitudinal courses: Amb, SW, FMP

7. The Challenge: Need to ensure curriculum is comprehensive
Complex curriculum – Need to know what others are teaching
Need to speak a common language
Creating curriculum de novo.No gaps or unwanted redundancies
Must address topics outside of basic sciences – LCME sociobehavioral economic topics
LCME accreditation must be able to show where a topic is covered in the curriculum

8. Curricular Mapping
Faculty Development sessions on curriculum development: Objectives –-- Assessments ---Content
Creation of Excel File Templates
Date
Format
Title
Objectives
Keywords
Took many sessions
Need to review
Keywords – limit to 5 per session + prepopulated columns for Sociobehavioral Economic topics and Biomedical Science topics
Overview of Oasis system

9. Process for entering data
Course Directors submit spreadsheets with information from their courses
CD’s then meet with library staff to align keywords with MESH terms – Common vocabulary
Information entered into OASIS

10. Challenges Culture change about curriculum development
Lack of understanding about key words (as opposed to all topics that were addressed)
Time commitment
Completion and monitoring of spreadsheets
Meetings to identify MESH terms
Process for entering data
Lack of experience with OASIS system

11. What will success look like?
Easily searchable curriculum
Clear reports for:
Alignment of session level objectives with course and institutional objectives
Identification of gaps and redundancies
Compelling data for success of our curriculum in meeting our goals.

12. Curriculum Mapping OHSU Today and Beyond
Presented by: Tomo Ito, MSEd
Oregon Health & Science University

13. Current curriculum structure 2 + 2
Current curriculum structure In 2014, new curriculum will be introduced.

14. Current mapping in E*Value is limited to…
Course title/duration
Event title
Event date/time/duration
Event facilitator
Instruction method
Assessment method
Resources
Handout/PowerPoint
Physician Competency Reference Set
OHSU SoM Competencies and Objectives
Course Objectives
OHSU has been using E*Value for the past 2 years.
Data coordinator is responsible for entering data.
Faculty course/clerkship directors link event objective to course objective. And Course objective to SoM Program Competency and objective.
Event Objectives

15. LCME ED-10 Hot topic: Cultural Competency Term search
What the report looks like:
LCME Ed10 asks schools to identify in which course/clerkship each content area is converged and the number of sessions/hours in required courses

16. Find where we teach Medical Knowledge #1
Schools can ensure that each competency/objective has been covered by running a report.
Show my where we cover Medical Knowledge, Objective #1, Identify the normal anatomy, histology, and embryology of major tissues, organs, and organ systems.

17. What’s Next?
Student Outcome Assessment Result from various systems (MCQ/Essay/OSCE /CPX /rating/narrative evaluation, etc.) must tie back into Curriculum Mapping
Each question can be tagged to either session level objective, course objective or UME program objective.
Outcome of students could potentially inform us content/instruction method, assessment method and assessment items. Could be utilize as a part of comprehensive review on course/clerkship.

18. Remediation Insight/ Self-reflection/ Life-long learning
Students miss Exam question 2
System pinpoints resources/modules to review
Students complete the modules and take additional exam on the topic
System recognizes attainment of competency/objective
Part or Whole
Insight/Self-reflection/life-long learning  between #1 and #2, we want to give learners opportunities to self-reflect. It’s a great opportunity for students to acquire the competency, Practice-based learning and improvement. So instead of the system pinpointing, students seek out information. Then, the system will be used as the secondary source.
Push/Pull
(pushing  we require them to remediate)
(Pull  we offer them but students have choice to either look at it or not)
Academic Advising A system will trigger notification to academic advisor automatically
At what level, students should self-reflect and seek out help.
Part or Whole?
Do we except students to re-do the whole or a part?
Academic Advising

19. Competency-based Education
Things to consider:
Competency-based Education
Not a time-based education
“Testing out” or “pre-assessment”
Indexing curriculum (Sequence or parallel)
Individualization of learning methods (not all learners learn the same way)
Quality attributes (how do we tag quality)
Assessment mapping
Lerner centered education. Differing experience and knowledge base.
True competency-based education looks at attainment of pre-defined benchmark/competency. If a student worked as a Nurse Practitioner prior to entering MD program, the level of knowledge will most likely differ from a student who just graduated from Sociology degree.
All learners do not learn the same way. All learners do not learn at the same pace.
Measuring attainment of competency becomes a focus.

20. A few more things… Educators’ teaching contribution
Voice recognition functionality
Automatic feed to E-Folio from curriculum mapping, evaluation, assessment results, procedure logs, EHR, API to LMS, registration system

21. Mapping Medical Education Curricula How to use the Data in Assessment
Rebecca S. Keller, PhD
Assistant Dean of Medical Education
Albany Medical College

22. Albany Medical College Curriculum
Organ System Based with Longitudinal Themes
Year 1: Normal Systems – biochemistry, molecular biology, physiology, anatomy, histology, nutrition, embryology, immunology, microbiology – Longitudinal: Clinical Skills, Evidence Based Medicine, Informatics, Health Care & Society
Year 2: Pathophysiology of Systems – pharmacology, infectious disease, pathology, clinical presentation, nutrition - Longitudinal: Clinical Skills, Evidence Based Medicine, Informatics, Health Care & Society
2 week Orientation Clerkship
Year 3: Required Clinical Clerkships – Medicine, Family Medicine, Pediatrics, Surgery, OB/GYN, Psychiatry - Longitudinal: Clinical Skills, Evidence Based Medicine, Informatics, Health Care & Society
Year 4: 5 Required Clerkships (Emed, Neurology, Critical Care, Acting Internship, Teaching to Learn), 4 Electives - Longitudinal: Clinical Skills, Evidence Based Medicine, Informatics, Health Care & Society

23. Mapping the Curriculum
Curriculum has been mapped using older techniques for many years – CurrMIT and in house paper files
Changed in 2008 with the database as developed at George Washington
Added data to the Curriculum Database starting in the Academic Year
We have data for the last 4 academic years – currently working on the database

27. AMC Goals for using a Database
Map our Graduation Objectives with our Theme/Clerkship Objectives for assessment
Searchable – to look for intentional overlap and for missing topics
Engage the faculty in curriculum assessment and development
All faculty, staff and students have access to the database
Theme Leader and Clerkship Directors were trained and provided data – one main person enters the data with back up (myself)
Identify both horizontal and vertical integration

28. Assessment of Data Combined Theme Leader/Clerkship Director Meetings
Case Based Discussion using data
CAT (Curriculum Assessment Team) – 3 year process for assessing all themes/clerkships
Use data from the curriculum database for assessment – along with theme leader/clerkship director reports, student evaluation, graduation questionnaire, STEP 1/2 scores, Match results, etc
Exams – use GEMS to map session learning objectives with exam questions – closing the loop on curriculum mapping

29. Challenges Time investment for faculty Faculty development
Writing learning objectives
Writing exam questions that are 2nd or 3rd order
Engagement of Students
Use the database for studying in Year 3/4, for board review

30. “Curriculum Mapping” at a School … in the Process of Curriculum Redesign
Lynn J. Romrell, Ph.D.
Florida State University College of Medicine
Associate Dean for Medical Education

31. “Events”

32. Curriculum Map – Event “Tags”
Required for Compliance with
LCME Standards ?
Medical School
Defined “Tags”
Physician Competency Reference Set
Competencies
Institutional Objectives
“Event”
Instructional Method
Assessment Method
Resources – Faculty,
Staff, Room, etc.
Time
MedBiquitous

33. Curriculum Map – Event “Tags”
Required for Compliance with
LCME Standards
Medical School Defined “Tags”:
Organ System
“Discipline”
“LCME Topics”
“Themes”
“Mission Specific”
Primary Topic(s)
- “Emphasis”
- “Time”
“Clinical Presentation” ?
Medical School
Defined “Tags”
Medical Education Competency Reference Set
Competencies
Institutional Objectives
“Event”
Instructional Method
Assessment Method
Resources – Faculty,
Staff, Room, etc.
Time
MedBiquitous

34. LCME Standards
ED-33 “There must be integrated institutional responsibility in a medical education program for the overall design, management, and evaluation of a coherent and coordinated curriculum.” ED-35 … “Provide evidence that the school monitors the content covered in the curriculum … ” ED-37 … “faculty committee … must be responsible for monitoring the curriculum, including the content”

35. $ $ $ $ Resources $ Balance of … Desire for innovation
Reality of costs and available resources $ $ $ $ $ $

36. Instructional and Assessment Methods
Methods of assessment impact “how and what students learn” Link assessments to competencies

37. What Do We Want to Map? What Event “Tags” Would Be Helpful?
Medical School Defined “Tags”:
Organ System
“Discipline”
“LCME Topics”
“Themes”
“Mission Specific”
Primary Topic(s)
- “Emphasis”
- “Time”
“Clinical Presentation”
Quantitative
Measure
Qualitative or
Frequency
Measures*
“Event”
*Items can be a assigned as a “Primary Topic”

38. Quantitation: “Primary Topic” Fields
Fields to be used to determine actual time of coverage
List up to four or five topics
Topics listed share the time equally unless level of emphasis is indicated
May indicate level of emphasis - 2X, 3X, or 4X
Topic
Emphasis
Time on Topic
Structure - Radiology
10 min
Respiratory System 4X
40 min
Shortness of Breathe

39. “Metadata” – Event Tags
Have been left in the “hands” of the medical schools
Specific tags can be used to emphasize integrative thinking rather than classical disciplines
“Structure” instead of “gross anatomy”
“Development” instead of “embryology”
“Cellular function”
“Pathophysiology”
“Translational Medicine”

40. Qualitative or Frequency Measures*
*Items can be a assigned as a “Primary Topic”

41. “FSUCOM Mission Specific”
“… responsive to community needs … through service to elder, rural, minority and underserved populations.”

42. “Patient Presentation” Tag
H. Mandin, University of Calgary
Faculty of Medicine
1. Abdominal Distension
2. Abdominal Mass
3. Abdominal Pain
4. Abnormal Chest X-Ray
5. Abnormal Heart Sounds
Etc.

43. “LCME Survey Topics in MSQ Part II”
“Hot Topics”
Biostatistics
End-of-Life Care
Domestic Violence
Global Health Issues
Palliative Care
Patient Safety
Etc.

44. “Themes” in the Curriculum
Nutrition
Biostatics/Epidemiology
Life stages
Ethical or Cultural Issues
Interprofessional Education
Population Health
Psychosocial Issues
Etc.

45. Standardized Patient Assisting the Learner

46. Curriculum Inventory Options for Mapping
Terri Cameron, Director of Curriculum Programs Association of American Medical Colleges

47. Curriculum Inventory Mapping
Expectations (Competencies, Learning Objectives, Outcomes Objectives)
PCRS
Program
Sequence Block
Event

48. Curriculum Inventory Mapping
Integration Sequence Block
Themes
Humanities
Informatics
Genetics
Public Health
Disciplines

49. Curriculum Inventory Mapping
Keywords
LCME ED-10 (Hot Topics)
Special Details of Instruction
USMLE Content Outline(s)
MESH / SnoMED / UMLS
Integration Terms

50. Curriculum Inventory Mapping
Documenting Keywords
Faculty
Checklists / Templates
Enter into database
Course Documentation
As metadata for teaching materials
Students
Documenting top 5 or 10 concepts
Checklist
Staff
Drafting from syllabus / other teaching materials
Meeting with / following teaching faculty

51. Curriculum Inventory Mapping
Curriculum Inventory Reports from Mapped Data
Competencies linked to content
How content is taught
How content is assessed
Where in curriculum content is taught
Required
Elective

52. Curriculum Mapping Case Study

53. Discussion Questions
Has your school embarked on a curriculum mapping process?
What are the goals for your mapping process?
What challenges have you faced or do you anticipate?
What solutions have you developed?
How did you train/educate/involve your faculty?
How are you assessing your mapping project?