1. Curriculum Inventory Administrators’ Group July 12, 2017
Terri Cameron, MA
Director, Curriculum Programs

2. Agenda AAMC Medical Education Cluster Update
Preparation for Upload
CI System Updates
Verification Report Review / Manual
Graduate Follow-up Survey
Medical school highlight: University of Washington
WGEA Presentation: A Deeper Dive into Curriculum Mapping: Creating a Searchable Curriculum Map
July CI in Context: Mapping Clerkship Metadata
August CI in Context: Update on the EPA Pilot
CI for CQI and Accreditation Task Force
CI Research Group
Next meeting: Wednesday, August 9, 1 pm ET

3. AAMC Medical Education Cluster Update
Senior Director, Digital Resources
In process
Senior Director, Research
“Director, Curriculum Programs”
Terri Cameron retiring

4. Preparation for 2016-2107 Upload
Uploads begin August 1; Portal closes September 30
EPA Documentation
Staging available now – new URL:
This does not affect use of the 'Production' CI Portal – that is still accessed at

5. Preparation for 2016-2107 Upload
Log into CI Portal
Upper right hand of
Choose ‘Manage Users’ tab to check names assigned to roles for:
Curriculum Dean(s)
Primary CI Admin(s)
Choose ‘Manage Sender’ tab to ensure correct vendor is selected
CI Portal User Guide:

6. Preparation for 2016-2107 Upload
EPA’s
No requirement to document EPA’s
For research purposes only
Best Practice for Documentation:
CoreEPA01_Gather a history and perform a physical examination
Timeline:
Document for upcoming AY:
Upload August 1, 2018

7. Preparation for 2016-2107 Upload
Resources
Documentation of Resources is NOT REQUIRED but STRONGLY ENCOURAGED
This allows us to report on use of Standardized Patients, Virtual Patients, Simulation, etc.
Resources MUST be documented using IDs, rather than text

8. Preparation for 2016-2107 Upload
Accuracy of Data
Spellcheck
Vendors tell us their systems have spellcheck, but we spend a great deal of time trying to find misspellings while running reports
Yearcheck 
2610 instead of 2016 can result in a very long course
Field length
Many of the text fields have a 4000 character length, which is about 8 pages of typed text; you might see if you have text that is cut off; and, if so, if you can delete text that does not relate to content

9. CLIN CLINIAL CLINICLAL CLINC CLINICALY CLINICLLY CLINCAL CLINICAN
Speaking the Same Language: Simple Improvements Clinic, Clinical, Clinician, Clinically
CLIN
CLINIAL
CLINICLAL
CLINC
CLINICALY
CLINICLLY
CLINCAL
CLINICAN
CLINICNAL
CLINCALL
CLINICIAL
CLINICSL
CLINCI
CLINICICALL
CLINIICAL
CLINCIAL
CLINICL
CLININCAL
CLINCICAL
CLINICLA
CLINIUCAL
CLINI

10. System Changes Implemented for 2016-2017
Migrated to Tomcat from Websphere to improve our testing processes and make future work easier.
Improved the processing of XML files over 19MB.
Files still need to be zipped, but should be processed more quickly now.
Improving communication with vendors re: data and system issues.
Verification Report organized by program expectation name, rather than order of appearance in the XML.
Changed tallies of assessment methods in the Summative and Formative Assessment tables so events are counted in one or the other.
Updated the timing for reminder notifications so schools won’t get reminders immediately after the opening of the season.
Fixed ‘loophole’ in academic level requirement: ALs MUST start with 1 and increase in single increments. This preventing corrupt files from being created.
Included language specifying the classification of CI data as Restricted.

11. Planned changes for 2017-2018 CI Upload Cycle:
Increase the length of the Academic Level Title
New AAMC Business Rule: Competency Object titles will be required.
New AAMC Business Rule: Event ID references cannot have spaces.
New AAMC Business Rule: The year value for date will need to be greater within 6 years of either direction of the report start and end dates.
New AAMC Business Rule: If Resources are documented, UIDs will be required (as they currently are for Instructional and Assessment methods).
New AAMC Business Rule: Special characters will not be accepted (the Business Rule will provide more information).
CompetencyObjects that are deeply placed in the competency framework will appear in the verification report.
The Curriculum Inventory will make use of the new System Access Management system at the AAMC, an important step in rewarding participation.

12. Verification Report
Please carefully check Academic Level Start and End dates on the first page of the report.
Calculated from earliest start date and latest end date of each AL’s Sequence Blocks.
Several of the summary tables are meant to replicate the LCME DCI tables. We hope to add additional tables for next year and reference the appropriate DCI table for each. Changes made for this year:
Program expectation table organized byname, rather than order of appearance in the XML.
Assessment methods are tallied in the Summative and Formative Assessment tables so events are counted in one or the other.

13. Verification Report Manual
Will provide:
Source and logic behind data and calculations in each table
Known issues that might cause unexpected results
Best Practices for documentation for expected results
Timeline:
First draft August 2017

14. Graduate Follow-Up Surveys
Are you seeing an increase in the number of Program Directors who are requiring a release from graduates?
How are you handling this?
AAMC continues to investigate possibility of implementing Graduate Follow-Up Survey in line with Matriculation and Graduation Surveys

15. University of Washington School of Medicine
Michael J. Ryan, MD
Associate Dean for Curriculum
Associate Professor, Department of Medicine
Division of Nephrology

16. >200 clinical training sites
WWAMI Partnership:
University of Washington
University of Wyoming
University of Alaska-Anchorage
Montana State University
University of Idaho
Gonzaga University
>200 clinical training sites

17. The five WWAMI states make up 28% of the U.S. land mass but contain only 3.4% of the country’s population.
“The five WWAMI states make up 27% of the U.S. land mass but contain only 3.3% of the country’s population.”
FYI: Suzanne has the WWAMI population percentage in a number of her slides.  The population number has increased from 3.3 to 3.4.
“The five WWAMI states make up 28% of the U.S. land mass but contain only 3.4% of the country’s population.”

19. Curriculum Management
Build or buy?
Integrate with Canvas (LMS)
3 time zones
Decision:
Online Portal for Advanced Learning Quality Management (OpalQM)
Created by Online Business Systems (OBS): privately held information technology and business consultancy
System created for the University of Manitoba

20. OpalQM: All Phases/components of curriculum
Session details
Objectives
Session type
Threads/theme tags
Content descriptors
Entered into Opal, then copied to Sites
Organized by Entering Class Phase/Block/Site
Each site schedules events in local time zone

21. PLO-BLO-SLO Structure

22. Number Of Formal Instructional Hours Per Course
Using Data: Pedagogy
Table 6.0-1  |  Entering Class Phase 1 Instructional Format
Number Of Formal Instructional Hours Per Course
Course
Lecture
Laboratory
Small Group
Patient Contact
Other*
Total
FCM: Clinical Immersion
4.00
1.00
33.08
38.08
FCM: Clinical Skills
75.25
85.25
160.50
FCM: Primary Care Practicum
133.00
Molecular & Cellular Basis of Disease
12.00
7.00
55.00
16.00
90.00
Invaders & Defenders
6.00
74.00
5.00
85.00
Circulatory Systems
16.08
58.00
145.08
Blood & Cancer
10.00
18.75
13.42
42.17
Energetics & Homeostasis
13.00
74.58
106.33
Mind, Brain & Behavior
3.00
30.17
104.00
153.17
Lifecycle
8.00
43.08
21.00
72.08
TOTAL
58.08
53.00
351.58
208.25
354.50
*Other Session Types Include:
Discussion, Large Group (>12)
Patient Presentation – Faculty
Simulation
Workshop

23. Use of Data: Gap Analysis

24. Assessment outcomes for a block

25. USMLE Content Descriptors – Search Results

26. Future Effort Develop model for clerkships
Map out all objectives, and use results to review and refine curriculum objectives

27. A Deeper Dive into Curriculum Mapping:
Creating a Searchable Curriculum Map
Michele M. Doucette, PhD
Assistant Dean of Integrated Curriculum
Daisy Talavera, MPA Integrated Curriculum Specialist
Helen Macfarlane, MA
Director of Education Technology

28. Overview what we will cover today…
Review purpose, goals, utility of curriculum mapping, including curriculum components commonly included in a curriculum map
Overview of the curriculum mapping framework implemented at the CUSOM
Demonstrate an interactive curriculum search tool, along with user-defined reporting capabilities, using Tableau®
Review faculty engagement in the mapping process

29. Curriculum Mapping utility…
Curriculum Search (Competency, Phase, Topics, Keywords, Session Type, etc.)
Curriculum Review
Gaps
Redundancies
Vertical/ Horizontal Integration
Course Curriculum Content Reports
Course Evaluations (Faculty Time)
LCME Accreditation Reporting/ Ongoing CQI
Student Portfolio

30. AAMC Curriculum Inventory (CI) Data Necessary for a Successful CI Upload
Sequence Blocks
Events
ALL Sequence Blocks: Courses, Clerkships, Electives
Duration
Expectations (Learning Objectives / Outcome Objectives / Competencies)
Academic Level
Duration in hours and / or minutes
Instructional Methods
Assessment Methods
Expectations (Learning Objectives / Outcome Objectives / Competencies)
Keywords
Source:

31. Curriculum Map Structure
Program Competencies
Graduation Requirements
(required elements of a learning experience)
Curriculum Structure
Course Goals
Overall goals of the course
Learning Objectives
Brief statements (what student can do at instruction completion)
Topics
Categories (big buckets)
(linked to curriculum components above)
Curriculum Content
MeSH Terms
Key terms
Learning Materials
Session materials

32. CUSOM Program Competencies
Competency
# Sub-Competencies
1. Medical Knowledge for Practice (MKP) 8
2. Patient Care (PC) 9
3. Practice-Based Learning and Improvement (PBLI) 6
4. Interpersonal and Communication Skills (IPCS) 11
5. Professionalism (P) 3
6. Systems-Based Practice (SBP) 4
7. Interprofessional Collaboration (IC)
8. Personal and Professional Development (P/PD)

33. CUSOM Topics
Adopted 90 new “Topics” aligned with LCME, GQ, USMLE, and CUSOM UME program:
Provided higher-level category to group curriculum
Aids in LCME reporting
Future use in student learning “Portfolios”

34. Keywords (MeSH Terms)
MeSH vocabulary similar in concept to keywords or (meta) tags
Unlike keywords, MeSH terms are “standardized” + hierarchical structure
When mapped to items in a curricular database (i.e., iLios), MeSH terms allow curricular content to be searched by a standardized set of terms
Love-hate relationship…

35. Curriculum Map Framework

36. Mapped Curriculum Components
Block/ Course
Curriculum Structure & Content
Course Goals (Objectives)
Phase
Session ID
Learning Objectives
Program Sub-Competencies
MeSH Terms
Session
Block/ Course ID
Session Title
Session Type
Date; Time; Duration
Faculty; Facilitators
Topic(s)
Source:

37. Curriculum Mapping what we use…
iLios – curriculum management
Curriculum components (course goals, session learning objectives, MeSH Terms)
Session data (course, phase, session type & title, date, time, duration)
MS Access, Excel – data management
Topics & Competency data
Compiled data (final output Excel worksheet)
Tableau® – data visualization
Software that allows you to visualize and interact with data

38. Demo Tableau®
Tableau®
Public
Desktop
Server/Enterprise

39. Unintentional Benefits of Mapping
Faculty begin to see their content through the lens of the Program Competencies
Faculty realize that they cover more topics than they originally thought
Mapping exercise prompts course directors think about how they want to structure their course
One-on-one mapping facilitates relationship- building with faculty
Faculty develop a better understanding of curriculum structure

40. Lessons Learned sustainability & validity…
Junk In ➡ Junk Out
No matter the CMS/ reporting system, it all comes down to gathering valid curriculum information
Dynamic, Ongoing
Establish a system/process to collect and update curriculum
Engaged Curriculum Deans, Faculty, Staff
It takes a village! Be patient, yet persistent.

41. Questions?

42. CI in Context: July 2017
Keywords Mapped to Clerkships in AL 3 and 4
Mapping Clerkship Metadata by  Cinda J. Stone, MEd University of Arizona, Phoenix

43. CI in Context: August 2017
Core EPA Pilot Update

44. CI for CQI and Accreditation (CICA)
The CICA’s most recent meeting on June 29th examined the Curriculum Inventory Verification Report and discussed its utility in addressing the DCI.
The summary tables were good starting points for determining where the curriculum management system data needed improvement.
The results in support of DCI responses must, as we’ve always intended, be modifiable to make them usable.
The actual values referred to in Other columns need to be provided in the output, though perhaps not the verification report.
Include academic level in the All Instructional Methods, Summative Assessment Methods, and Formative Assessment Methods tables to aid review.

45. Curriculum Inventory Research Group
Preparing for October Retreat

46. Next meeting: Wednesday, August 9, 1 pm ET
(Second Wednesday of each month, 1 pm ET)
Registration Links posted in Training and Resources section of
Please send agenda items to