1. Best Practices for Using Your Curriculum Management System
Brady Janes, PhD, University of Nevada
Kevin Souza, PhD, University of California – San Francisco
Paul Standley, PhD, University of Arizona
Cinda Stone, MEd, University of Arizona
Terri Cameron, MA, Association of American Medical Colleges
Walter Fitz-William, MPP, Association of American Medical Colleges

2. Best Practices for Curriculum Management Systems
Schools invest time, finances, energy, committee time, and human resources in these Curriculum Management Systems.
This investment should result in more informed curriculum administrators, committees, faculty, staff, and students using reports that:
Inform curriculum decisions, continuous quality improvement and benchmarking efforts;
Track competencies, key terminology, and themes/special emphases.

3. Objectives
At the conclusion of this session, participants will be able to:
Review features of Curriculum Management Systems
Explain the types of data that are important to the success of a Curriculum Management System
Discuss how Curriculum Management Systems can support curriculum committee discussions, administrative decisions, and student preparation for the curriculum
Share opportunities and challenges from other medical schools
Demonstrate how local curriculum data can be benchmarked with national data

4. Small Group Discussion Questions:
What are Best Practices for the types of data that should be collected?
What are Best Practices for collecting curriculum data?
What are Best Practices for using curriculum data to support Curriculum Committees, Administrators, Faculty, Students, and Staff?
What are Best Practices for using curriculum data to support benchmarking, CQI, curriculum review/renewal, and accreditation?
What features are necessary in a Curriculum Management System to support these Best Practices?

5. Using Heat Maps
A helpful report created for Medical Education Committees
Brady Janes
University of Nevada, Reno
School of Medicine

6. Hot = got it covered, strength, appropriate or over use
Snapshot of what we are teaching, how we are teaching it, how we are assessing it, and why we are teaching it.
Hot = got it covered, strength, appropriate or over use
Cool = not covered, weakness, not appropriate or under use
This approach provides a visual representation of current practice.
Questions to ask when looking at heat maps –
What do the color shades mean?
Red could mean the area is frequently covered, this area is a strength, this area is appropriate for the session, and or this area is “excessively” covered.
Blue could mean the area is missing, this area is weak, this area is not appropriate for the session, and or this area is “marginally” covered.

7. Hot/Cool Mapping Example of what we are teaching (Content)
CA01 Anatomy
CA02 Behavioral Science
CA03 Biochemistry
CA04 Biostatistics
CA05 Cell Biology
CA06 Clinical Skills
CA07 Embryology
CA08 Epidemiology
CA09 Ethics
CA10 Evidence-based Medicine
CA11 Genetics
CA12 Histology
CA13 Immunology
CA14 Microbiology
CA15 Neuroscience
CA16 Nutrition
CA17 Organ System Pathophysiology
CA18 Pathology
CA19 Pharmacology
CA20 Physiology
CA21 Radiology
CA22 Population Health
Block 01: Foundations and Principles of Medical Science I 8 11 17 10 4 6 31 1 2 3 13
Block 02: Cardiovascular, Respiratory and Renal Systems I 9 5 7 45
Block 03: Gastrointestinal, Endocrine and Reproductive Systems I 19 21 22
Block 04: Musculoskeletal and Integumentary Systems
Block 05: Nervous System and Human Behavior I 15 47
POMF: Practice of Medicine - Fall
POMS: Practice of Medicine - Spring
Totals 73 61 42 18 85 14 37 32 52 26
For example, this snapshot of a year 1 content (what we are teaching) heat map shows the courses on the left, the content areas on the top, and the frequency or tagging of that content within the courses.
In looking at this map as an overview for the year 1 curriculum, a committee member might note that the year 1 concentration is on anatomy and physiology, clinical skills, behavior science, and biochemistry. But, biostatistics, epidemiology, immunology, microbiology, and nutrition are areas of content that are not covered in the year 1 curriculum.
Although this is not a complete representation of a content heat map – as these content areas may be covered in years 2, 3, and or 4, hopefully you can imagine that this tool opens the door for discussions about placement and depth of content coverage throughout the curriculum.

8. Hot/Cool Mapping Example of how we are teaching (Methods)
Moment name
Case-Based Instruction/Learning
Clinical Experience - Ambulatory
Clinical Experience - Inpatient
Concept Mapping
Conference
Demonstration
Discussion, Large Group (>12)
Discussion, Small Group (≤ 12)
Games
Independent Learning
Journal Club/ EBM presentation
Laboratory
Lecture
Mentorship
Patient Presentation - Faculty
Patient Presentation - Learner
Peer Teaching
Preceptorship
Problem-Based Learning (PBL)
Reflection/Writing
Research
Role Play/ Dramatization
Self-directed Learning
Service Learning Activity
Simulation
Team-Based Learning
Team-Building
Tutorial
Ward Rounds
Workshop
Year 1
Block 01: Foundations and Principles of Medical Science I 16 1 3 4 5 67 12 2
Block 02: Cardiovascular, Respiratory and Renal Systems I 9 8 6 62
Block 03: Gastrointestinal, Endocrine and Reproductive Systems I 10
Block 04: Musculoskeletal and Integumentary Systems 27
Block 05: Nervous System and Human Behavior I 13 54
POMF: Practice of Medicine - Fall 7
POMS: Practice of Medicine - Spring
Totals 39 14 31 57 44
285 11
Year 2
Block 06: Foundations and Principles of Medical Science II 42
Block 07: Whole Body
Block 08: Nervous System and Human Behavior II 80
Block 09: Cardiovascular, Respiratory and Renal Systems II 30
Block 10: Gastrointestinal, Endocrine, and Reproductive Systems II
ACSP: Advanced Clinical Skills - Preceptorship
ACS: Advanced Clinical Skills
Year 2 post Step
CPC: Context of Patient Care 43 40 29 25
268
Year 1-2 79 21 41 86 69
553
This snapshot of a year 1 teaching methods heat map shows the courses on the left, the teaching methods on the top, and the frequency or tagging of that method within the courses.
Discussion that can arise from looking at this heat map is it seems much of the instruction is done using lectures, which may be appropriate, but are there other methods that may be more effective?

9. Hot/Cool Mapping Example of how we are assessing
List - Assessment Methods
Clinical Documentation Review
Clinical Performance Rating/Checklist
Exam - Institutionally Developed, Clinical Performance
Exam - Institutionally Developed, Oral
Exam - Institutionally Developed, Written/ Computer-based
Mini CEX
TBL
Narrative Assessment
Oral Patient Presentation
Participation
Peer Assessment
Portfolio-Based Assessment
Practical (Lab)
Research or Project Assessment - Oral or Written
Self-Assessment/Quiz
Written Assignment
Year 1
Block 01: Foundations and Principles of Medical Science I 18 3 6 1 8
Block 02: Cardiovascular, Respiratory and Renal Systems I 17 5 4 7
Block 03: Gastrointestinal, Endocrine and Reproductive Systems I 2
Block 04: Musculoskeletal and Integumentary Systems 9
Block 05: Nervous System and Human Behavior I 14
POMF: Practice of Medicine - Fall
POMS: Practice of Medicine - Spring 11
Totals 95 13 23 36
Year 2
Block 06: Foundations and Principles of Medical Science II
Block 07: Whole Body 10
Block 08: Nervous System and Human Behavior II
Block 09: Cardiovascular, Respiratory and Renal Systems II
Block 10: Gastrointestinal, Endocrine, and Reproductive Systems II 12
ACS: Advanced Clinical Skills
ACSP: Advanced Clinical Skills - Preceptorship
Year 2 post Step
CPC: Context of Patient Care 71 31
Years 1-2
166 40 67
This snapshot of years 1 and 2 assessments heat map shows the courses on the left, the types of assessments on the top, and the frequency or tagging of the assessment types within the courses.
Discussion around this map could be whether or not the assessments capture student performance adequately and diversification of assessment methods. Are there other types of assessment that may be appropriate?

10. Hot/Cool Mapping Example of why we are teaching
BLOCK 1 WEEK 1
MEPO 1: Knowledge for Practice
MEPO 2: Interpersonal and Communication Skills
MEPO 3: Professionalism
SESSION
TOTALS
1.1
1.2
1.3
1.4
1.5
1.6
2.1
2.2
2.3
2.4
2.5
2.6
2.7
3.1
3.2
3.3
3.4
3.5
3.6
1/1 m: A Day in the Life 4 1
1/2 m: Introduction to Block 1
1/3 r: Introduction to Laboratory Medicine; Interpreting Clinical Tests 2
1/4 m: EBM-Using & Evaluating Scientific Literature 3
2/1 r: Introduction to Imaging
2/2 r: Introduction to Human Behavior
2/4 m: EBM-Using Information Resources
3/1 r: Introduction to Pharmacology- Prescription writing
3/2 r: BPSS model
4/1 r: Genomes, Evolution, and Genetics in Medicine; DNA replication, Recombination, Repair
4/2 r: Chromosomes and Heredity
4/3 r: Molecular Genetics
4/4 m: Strategies for Small Group learning and Clinical Problem Solving
5/1 r: RBC/WBC Terminology and CBC
5/2 r: Basic principles of Mendelian Inheritance, Pedigree Analysis
5/3 r: Introduction to Biochemistry Chemical Reactions
5/4 m: EBM-Reading Scientific Papers
And, finally, this is a snapshot of an instructional week heat map with the instructional sessions on the left, the medical education program objectives on the top, and the frequency or tagging of those objectives for each session.
This map shows the “Why”. It provides evidence that the instructional session supports one or several of the medical education program objectives, which, if a session is not map – able to a program objective, then the questions are – Why are we teaching it or Are we missing something in our program objectives?
Overall, these maps provide committee members a visual overview of a curriculum in order to reflect on the content, instruction, assessment, and program objectives.

11. Pre-Populating Standard Course Reports
Kevin H. Souza, MS, University of California – San Francisco

12. CMS Tagging - Current Instructional method; learning resource(s)
Topics; chosen based on LCME standards, GQ items, school values, USMLE exam student/school reports
MeSH terms
Infection, Immunity and Inflammation block; Antiviral 1 lecture

13. CMS Tagging – Current & Aspirational
Instructional method; learning resource(s)
Topics; chosen based on LCME standards, GQ items, school values, USMLE exam student/school reports
MeSH terms
UCSF 49 (curated collection of 49 core syndromes & disorders that will anchor students' journeys through the Bridges Curriculum)
Clinical case patient characteristics (to address diversity and stereotyping issues)

14. Standard Components of Course Report
Student ratings of sessions and instructors, & student comments; assigned by communication between CMS and E*Value
Teaching hours report from CMS
Monitoring days with instructional time greater than guidelines

15. Standard Components of Course Report
Teaching hours report from CMS
Topic summaries by total hours and subdivided by instructional method
Provides info about integrated sessions

16. Standard Components of Course Report
Teaching hours report from CMS
Details in separate report

17. Course Directors Reflect on Course Report
Reflecting on the Topic Distribution and Evaluation Summary Reports, identify one to three areas that represent a gap or redundancy that you will correct in the coming year.
Referring to the Evaluation Summary Report, comment on how you will reduce the number of days when lecture equaled or exceeded 3 hours.
Referring to the Evaluation Summary Report, comment on how you will reduce the number of days when total class time exceeded 4 hours. 

18. Applying Best Practices from Years 1 and 2 to Clerkship Years
Paul Standley, PhD, University of Arizona
Cinda Stone, MEd, University of Arizona

19. Applying Best Practices from Years 1 and 2 to the Clerkship Years
Best Practices learned from Year 1 and 2 to map Clerkship Years
Reinforced the need for setting expectations/threshold with faculty for mapping/tagging – meetings with Directors
Allowed for Clinical Experience Clerkship mapping of:
Procedures (Px) and Diagnosis (Dx)– developed objectives and tagged with Disciplines and CCT’s
Supported the importance of CMS flexibility for different program/course structures
Don’t change the structure of course to fit CMS rather use CMS options (i.e. adding increments of time for instructional methods) to fit the structure

20. Data gathered for success of CMS
Session information (duration, lecturer, instructional method)
EPO, Block, and Session learning objectives
Comprehensive Curricular Topics (CCT), and Disciplines
Breakout of instructional method into increments of time

21. CMS reports are utilized for:
Identifying gaps –
Are all block/clerkship objectives utilized?
Overlook CCT and/or Disciplines?
Ensure students of quality improvement and growth
Enhance theme and topic curriculum

22. CMS reports are utilized for:
What instructional methods are being applied and what sessions can be flipped
Discussion in committees to facilitate change based on reporting outcomes - Px and Dx as sessions, having objectives, and tagging them with disciplines and CCTs.
Collect data for accreditation

23. Benchmarking Local Curriculum Data with National Data
Terri Cameron, MA, Association of American Medical Colleges
Walter Fitz-William, MPP, Association of American Medical Colleges