Curriculum Inventory Administrators’ Group August 9, 2017 Terri Cameron, MA Director, Curriculum Programs

Agenda AAMC Medical Education Cluster Update CI 2016-2017 Upload CI Kick-off Verification Report Review / User Guide Planning for Veronica Catanese, MD, LCME Secretariat, CIAG Presentation Curriculum Directory Graduate Follow-up Survey Medical school highlight: University of South Dakota August CI in Context: Update on the EPA Pilot September CI in Context: Tracks in US Medical School Curricula CI for CQI and Accreditation Task Force CI Research Group Next meeting: Wednesday, September 13, 1 pm ET

AAMC Medical Education Cluster Update Senior Director, Digital Resources In process Senior Director, Research “Director, Curriculum Programs” Terri Cameron retiring

CI 2016-2107 Kick-off Apologies for: August 1 Kick-off e-mail and immediate reminder Staging password expirations Affects only those schools who vendors use School Staging credentials to log in to School Staging Portal for testing Solution: School logs into School Staging Portal with existing credentials; system assists with setting up new password; School provides Vendor with new password Staging credentials are not tied to an individual person’s AAMC ID – they are specific to an institution, not a person and are only valid in the Staging environment System error to those who tried to upload on August 1 (resolved)

CI 2016-2107 Upload Uploads began August 1; Portal closes September 30 3 Schools Verified: University of Nebraska West Virginia University Yale University 2 Schools Reviewing 7 Schools In Process 3 Schools Declined Staging available – new URL: http://apps.staging.aamc.org/ci-school-web This does not affect use of the 'Production' CI Portal – that is still accessed at www.aamc.org/cir

Verification Report Please carefully check Academic Level Start and End dates on the first page of the report. Calculated from earliest start date and latest end date of each AL’s Sequence Blocks. Several of the summary tables are meant to replicate the LCME DCI tables. We hope to add additional tables for next year and reference the appropriate DCI table for each. Changes made for this year: Program expectation table organized by name, rather than order of appearance in the XML. Assessment methods are tallied in the Summative and Formative Assessment tables so events are counted in one or the other.

Verification Report User Guide Primary Methods of Instruction: This table lists all non-Clerkship SBs and shows how all of the Primary Instructional Methods linked to Events in those SBs map to the list of LCME DCI Educational Methods (see Crosswalk) and provides a total number of hours for each SB based on the duration of the events in each SB mapped to those Instructional Methods. Academic Level is based on the AL referenced by each SB in the CI Upload. Primary Methods of Instruction: Clerkships: Sequence Blocks will only be included in this table if they are tagged as clerkships. The number of weeks is calculated by dividing the number of days for the SB by 5. The hours of formal instruction are calculated by totaling the durations of all of the events with Primary Instructional Methods linked to the SB as those Instructional Methods are mapped to the list of LCME DCI Educational Methods (see Crosswalk).

Verification Report User Guide Methods of Assessment: This table lists all non-Clerkship SBs and shows how Summative Assessment methods linked to Events in those SBs are mapped to the list of LCME DCI Educational Methods (see Crosswalk). The Total Number of Exams is calculated by totaling the number of Events for that SB that have Summative Assessment Methods but do not have Instructional Methods linked to them. The Formative Column is checked If at least one event contains one or more Assessment Methods tagged as “Formative.” Academic Level is based on the AL referenced by each SB in the CI Upload. Methods of Assessment: Clerkships: Sequence Blocks will only be included in this table if they are tagged as clerkships. Summative Assessment methods linked to Events in those SBs are mapped to the list of LCME DCI Assessment Methods (see Crosswalk).

Planning for Veronica Catanese, MD, LCME Secretariat, CIAG Presentation Introduction LCME Update What questions would you like for her to answer? What is LCME position on AAMC Core EPAs? Faculty access to curriculum management system to view curriculum mapping?

Curriculum Directory Last produced from CurrMIT data in 2009 (available at www.aamc.org/cir under Training and Resources) Collection of tables related to US and Canadian Curricula Many aggregate, and those are represented in existing CIR Individual schools listed for Grading and USMLE policies, course listings Individual schematics for medical schools Possible solution to Schools’ need for data vs declination to have institution name attached to data in charts Proposed process: Add new chart data to Verification Report (first approval) Send each school a report showing data that will be included in Curriculum Directory Charts for final approval Produce Curriculum Directory based on agreed-upon charts

Curriculum Directory Charts including Institution Name Grading Systems (pull from LCME AQ II) USMLE Policies (pull from LCME AQ II) Sequence Blocks by Academic Level with Number of Weeks? Use of Instructional Methods (no hours/percentage/weight)? Use of Assessment Methods (no hours/percentage/weight)? Use of Resources (no hours/percentage/weight)? What else would be useful?

Graduate Follow-Up Surveys Are you seeing an increase in the number of Program Directors who are requiring a release from graduates? How are you handling this? AAMC continues to investigate possibility of implementing Graduate Follow-Up Survey in line with Matriculation and Graduation Surveys

Using the Curriculum Inventory to Prepare for LCME Re-Accreditation Shane Schellpfeffer, EdD Director of Evaluation and Assessment Curriculum Inventory Administrators’ Group, August 9, 2017

Introduction The University of South Dakota Sanford School of Medicine (USD SSOM) will have a full re-accreditation visit in September of 2017 We used the Curriculum Inventory (CI) for data mining in order to assist with the completion of the Data Collection Instrument (DCI) as outlined by the LCME

How We Use(d) Our CI SSOM uses One45 for our curriculum management system Data are entered centrally by ~2 individuals Analyze the Curriculum Verification Report Search for topics identified by the student ISA as “areas needing a deeper look” Example: Behavioral health in the basic science curriculum

Curriculum Verification Report Analysis Tracking course director’s session objectives Mapping course director’s session objectives Tracking course duration (hours) Mapping assessment methods (formative and summative) AY 2015-2016 CVR was 477 pages

Data Mining LCME Standard 6: Competencies, Curricular Objectives, and Curricular Design Used to mine data for multiple elements under this standard Focus: Instructional formats and duration (hours) of courses and/or clerkships Examples: Element 6.1; Tables 6.0-1, 6.0-2 & 6.4-1

Data Mining (cont.) LCME Standard 7: Curricular Content Used to mine data for multiple elements under this standard Focus: Curriculum content mapping to SSOM Competencies and/or LCME Hot Topics Examples: Elements 7.4 & 7.5; Tables 7.6-1 & 7.6-2

Data Mining (cont.) LCME Standard 8: Curricular Management, Evaluation, and Enhancement Used to mine data for some elements under this standard Focus: Overall monitoring and curriculum management Examples: Elements 8.1, 8.2 & 8.3--required to search for terms “acid-base balance” and “health care financing”

Data Mining (cont.) LCME Standard 9: Teaching, Supervision, Assessment, and Student and Patient Safety Used to mine data for a few elements under this standard Focus: Assessment methods for each stage of the curriculum Examples: Element 9.5; Tables 9.0-1, 9.0-2 & 9.0-3

Possible Additional Uses Additional Data Mining: Element 1.1: Strategic Planning and Continuous Quality Improvement Element 3.2: Community of Scholars/Research Opportunities Element 3.3: Diversity/Pipeline Programs Either identified as places to bolster our data input or start collecting data

Conclusions The curriculum inventory is an indispensable tool to use for data mining Entering accurate and thorough information will save time down the road (garbage in, garbage out) Questions??? Email: Shane.Schellpfeffer@usd.edu

CI in Context: August 2017 Core EPA Pilot Update Kim Loomis, MD Vanderbilt University

CI in Context: September 2017 Tracks in US Medical Curricula Sheila Crowe, EdD University of Oklahoma

CI for CQI and Accreditation (CICA) CICA did not meet in July.

Curriculum Inventory Research Group Preparing for October 4 Retreat with new Sr. Director for Research

Next meeting: Wednesday, September 13, 1 pm ET (Second Wednesday of each month, 1 pm ET) Registration Links posted in Training and Resources section of www.aamc.org/cir Please send agenda items to tcameron@aamc.org