Curriculum Inventory Administrators’ Group August 12, 2015 Terri Cameron, MA, Director, Curriculum Programs

Audio:  You will hear the audio through your computer speakers. Please make sure your computer speakers are on and the sound is turned up.  If you still have no sound once the webinar starts, please click on the audio broadcast icon ( ) located in the Participants Panel on the right hand side of your screen. Housekeeping  Please use the Q&A panel located on the right hand side of your screen to submit your questions throughout the webinar. Send to All Panelists. If you experience any technical or audio issues during the webinar, please send a message through the Chat panel to AAMC Meetings. Questions :

Please use the Chat panel located on the right hand side of your screen to submit your questions. Send to All Panelists. Questions

CI Upload Report Updated CI Website August CI in Context: Medical Schools Offering International Electives: Average Number of Medical Students in International Electives (Linda Gwinn and Irma Frank, Georgetown University) Featured Reports Medical Schools Offering International Electives: Number of Schools Medical Schools Offering International Electives: Curriculum Years of International Electives Curriculum Mapping Analyses CI Standardized Vocabulary Revision MedAPS Advisory Board Retreat Vendor Retreat CI at AAMC Medical Education Meeting and Learn Serve Lead Medical School Highlight: University of Texas Medical Branch – Galveston (Christine Ford) Other updates from participants Next meeting: Wednesday, September 9, 1 pm ET Agenda

CI Upload Report As of August 12: Verified: 3 University of North Carolina George Washington University University of Nebraska In Process: 18 Goal: 130

CI Website Update New left side navigation Updated content Updated Weekly Organized by User access: Curriculum Administrators System Developers Sign in to CI Portal still in upper right corner View status updates Download / Reject / Verify Verification Report

CI Website Update Alphabetical list of reports, as well as category groupings Use CTRL+F to quickly find a report Content Report Request link to be added soon Use browser ‘Back’ option to return to previous view Navigation to Report Category no longer available

Curriculum Content Curriculum Design and Policies Instruction and Assessment Programs and Institutions Alphabetical Listing of Reports Request a Report Curriculum Inventory & Reports (CIR) is comprised of graphical interpretations of both aggregate and historical curriculum-related data. Reports are based on two sets of data: The AAMC Curriculum Inventory, which includes detailed curriculum data uploaded each year by US and Canadian medical schools. The Liaison Committee on Medical Education (LCME) Annual Questionnaire, Part II, which includes curriculum policy and structure data from all U.S. AAMC-member medical schools. AAMC-member medical schools Reports are categorized into four categories, or may be viewed and searched as an alphabetical list. Content Reports that include where and how specific content (e.g., Genetics, Geriatrics) is taught and assessed may be requested by clicking on the ‘Request a Report’ button. Summary results for searches will be sent to requestors within three business days. Suggestions for additional CIR charts may be submitted to Featured Report

CI in Context: August Average Number of Medical Students in International Electives Linda Gwinn and Irma Frank, Georgetown University)

Featured CI Report Related Reports Electives: Average Number of Weeks That Students in the Most Recent Graduating Class Spent Taking Electives at Another Institution Electives: Medical Schools Having a Policy About the Maximum Number of Weeks or Maximum Number of Elective Rotations a Medical Student can take at Another Institution Electives: Medical Schools Offering International Elective Courses Electives: The Maximum Number of Weeks and the Maximum Number of Elective Rotations a Medical Student Spend Taking Electives at Another Institution

Featured CI Report Related Reports Electives: Average Number of Weeks That Students in the Most Recent Graduating Class Spent Taking Electives at Another Institution Electives: Medical Schools Having a Policy About the Maximum Number of Weeks or Maximum Number of Elective Rotations a Medical Student can take at Another Institution Electives: Medical Schools Offering International Elective Courses Electives: The Maximum Number of Weeks and the Maximum Number of Elective Rotations a Medical Student Spend Taking Electives at Another Institution

Curriculum Mapping Analyses 1. How did you develop your Curriculum Mapping terminology set? (e.g., Faculty Committee, Curriculum Committee (as a whole), Curriculum Administrators, External List(s), Student Committee, Other (what) 2. How many terms/concepts does your Curriculum Mapping terminology set include (e.g., Less than 100, , , More than 300) 3. How long did it take you to develop your Curriculum Mapping terminology set? (e.g., Less than a Year; One Year, Greater than One Year) 4. How often do you review your Curriculum Mapping terminology set? (e.g., Annually, Bi-Annually, Other (what)) 5. Do you have reports that reference your Curriculum Mapping terminology set? Yes / No 6. How often do you review reports that reference your Curriculum Mapping terminology set? (e.g., Annually, Bi-Annually, Other (what))

Curriculum Mapping Analyses Thanks to: Robin Rivest, Oakland University (Beaumont) for stimulating this conversation, and Susan Masters, University of California – San Francisco Tomo Ito, Oregon Health Sciences University Cyndi Lybrand, Eastern Tennessee State University (Quillen) John Mahoney, University of Pittsburgh Kevin Krane, Tulane University for providing data from their schools to help us begin this conversation and documentation.

Curriculum Mapping Analyses 1. How did you develop your Curriculum Mapping terminology set? (e.g., Faculty Committee, Curriculum Committee (as a whole), Curriculum Administrators, External List(s), Student Committee, Other (what) small group of curriculum leaders after a staff analyst went through the sources – NBME score reports, LCME, GQ to pick out topics and then align them for overlap Curriculum Developers (since we have new curriculum) and LCME’s list Curriculum Administrators drafted and Curriculum Committee approved as a whole Developed by faculty leaders of Curriculum Committee plus deans for med-ed We didn’t think about it well enough when we started, so we used terms we traditionally used where we knew what terms meant but others may have defined things differently. We are now using the Medbiquitous definitions.

Curriculum Mapping Analyses 2. How many terms/concepts does your Curriculum Mapping terminology set include (e.g., Less than 100, , , More than 300) 69 Less than – current debate / discussion on adding four levels of USMLE Content Outline Just under 100, plus a set of SOM objectives, approximately 230 Have not counted them

Curriculum Mapping Analyses 3. How long did it take you to develop your Curriculum Mapping terminology set? (e.g., Less than a Year; One Year, Greater than One Year) A few months Less than a year Less than a Year [initial version] ongoing CQI and seeking best practices (how much is enough???) Less than a year

Curriculum Mapping Analyses 4. How often do you review your Curriculum Mapping terminology set? (e.g., Annually, Bi-Annually, Other (what)) We haven’t done it for a few years – will for our new curriculum Have not reviewed them since they are brand new, but most likely annually moving forward Other (driven by program level objectives (recent adoption of PCRS) and other ‘system’ review of requirements [LCME Elements, DCI, AAMC GQ etc.) and our baseline on assessment tagging / framework) No defined schedule Only as we prepare for uploads

Curriculum Mapping Analyses 5./6. Do you have reports that reference your Curriculum Mapping terminology set? Yes / No; How often do you review reports that reference your Curriculum Mapping terminology set? (e.g., Annually, Bi-Annually, Other) Yes; Each course gets it with their report. We haven’t done a huge review. Much more likely to use the topics to see where things are being taught. Yes; Moving forward, annually. Yes; Working to get Vendor to include within summary reporting [use across multiple reporting needs. Review reports at multiple levels within the systematic review / component within Program Evaluation / Continuous Quality Improvement mechanisms, especially important to close the loop at / with the assessment level Yes; Annually and as needed No; We have not gone back to review until recently when we went with the Medbiquitous terms which will affect things for the future.

Animation Audience Response System Audio Cadaver Clinical Case Clinical Correlation Distance Learning - Asynchronous Distance Learning - Synchronous Educational Technology Electronic Health / Medical Record (EHR/EMR) CI Standardized Vocabulary Revision: Resources Film/Video Key Feature Mannequin Medical Images Mobile Application Patient – Receiving Clinical Care Patient - Teaching Plastinated Specimens Printed Materials (or Digital Equivalent) (see Written or Visual Media) Real Patient (see ‘Patient’) Scenario Searchable Electronic Database Standardized/Simulated Patient (SP) Task Trainer Ultrasound Virtual Patient Virtual Reality Virtual/Computerized Laboratory Wet Laboratory Written or Visual Media

CI Standardized Vocabulary Revision: Resources Will be posted to iCollaborative for comment (September ?) Will be presented/discussed at fall meetings Should be used for upload (too many changes to implement this year) Updated CI Standardized Vocabulary (with ID’s for resources) will be released in Fall 2015 for required implementation with CI Upload.

MedAPS Advisory Board Annual Retreat Board Updates: Thank you to Christopher Candler, University of Oklahoma, for four years of service to the Board Welcome to Giulia Bonaminio, Kansas University Medical Center, who begins her term with the October meeting of the Board

MedAPS Advisory Board Annual Retreat Focus Groups: ASSET (Colleen Growchoski) Curriculum Inventory (Hugh Stoddard) CI to ASSET Pre-population (Jennifer Christner) ASSET OMI (Heather Hageman) CI: GME (Kevin Krane) CI Reports (John Mahoney) NBME Collaboration (Maryellen Gusic) CI Standardized Vocabulary (Susan Masters) CI: IPE (Heather Hageman) PCRS (Paul Standley) Documenting Clerkship and Electives (Nancy Hueppchen) CI ‘Sticky Issues’: Primary Instructional Method, Formative / Summative Assessment, and Documentation of Multiple Rotations / Sites (Hugh Stoddard)

Vendor Retreat Discussion Issues: Competency mapping at the ‘top’ level (e.g., PCRS to CanMEDS, AACOM, ACGME Potential changes to the CI Data Exchange Standard Mapping, uploading, and reporting on Resources ASSET Processes CI to ASSET pre-population Potential new keyword lists Documenting clerkships and small groups Linking curriculum data to assessment data Collection of GME data

CI at AAMC Medical Education Meeting and Learn Serve Lead 30-Minute MedAPS Session Date/time not yet announced One-on-one sessions (Registration form at Exhibit Hall Focus Groups Dates/times not yet announced

Curriculum Mapping Curriculum Inventory Administrators Group Webinar University of Texas Medical Branch Christine Ford, Medical Educator Specialist August 12, 2015 School of Medicine

Curriculum Mapping Process 2 School of Medicine When we started mapping 2013, completely from scratch Where our map data comes from Course/clerkship directors Syllabi Blackboard Annual update process How that data gets in the map New Innovations Direct entry by me, checked by directors

Curriculum Mapping Process Cont. What data goes in the map All required courses and clerkships, minimester Electives/selectives; optional clerkship Varying level of detail: for electives Objectives: PCRS Program Course Events: Instructional method, assessment method, keywords, resources Files: lecture objectives, PBL PowerPoints, etc 27 School of Medicine

Interface demonstration – Course overview Basic structure: Course Events & Objectives 28 School of Medicine

Interface demonstration – Objective linking Linking program objectives to course objectives 29 School of Medicine

Interface demonstration – Event components 30 School of Medicine Objectives, Instructional Methods, Assessment Methods

Interface demonstration – Event components 31 School of Medicine Attachments, Resources, and Keywords

How We Use Our Map LCME Visit Answering questions from committee about where certain topics/objectives are covered New Innovations helped us generate reports quickly during the visit LCME “Hot Topics” List: already had these keywords in map before visit, helped us identify areas they might ask about and prepare 32 School of Medicine

How We Use Our Map Cont. Gaps & Redundancies Committee Subcommittee of Curriculum Committee Map is consulted as part of determining where we have gaps/redundancies Monthly topic surveys to course directors part of our regular process Course Planning Where do we teach X? At what level? 33 School of Medicine

Challenges Keywords MeSH terms: preloaded in New Innovations, too many to be practical Need for subject matter expertise What topic lists are other schools using? 34 School of Medicine

Questions? 35 School of Medicine

Use ‘Chat’ to tell us what’s going on with you! Other updates from participants

CI Portal opens August 1 Closes September 30 CI Portal Manual available at guide.pdf guide.pdf Submit early to allow time for review of Verification Report No changes to Portal or VR Focus on summary tables – use detail to review potential discrepancies in summary tables

Wednesday, September 9, 1 pm ET (Second Wednesday of each month, 1 pm ET) Registration Link: ex.com/aamc1/onstage/g.php?MTID=eeed77e d6d8406a097f54 ex.com/aamc1/onstage/g.php?MTID=eeed77e d6d8406a097f54 Please send agenda items to Next meeting: