Terri Cameron, MA Director, Curriculum Programs AAMC May 23, 2016 Documenting Clerkships and Electives Terri Cameron, MA Director, Curriculum Programs AAMC May 23, 2016

Objectives Discuss the types of data that should be entered for Clerkships and Electives Identify sources for gathering Clerkship and Electives data Explain options for entering one iteration of a clerkship Describe how nested sequence blocks might be used in documentation of Clerkships and Electives Describe options for entering ‘Colleges,’ ‘Academies,’ ‘Bootcamp,’ ‘Sub-Internships,’ and other student/content groupings options Review examples of how schools are documenting Clerkships and Electives Terri www.aamc.org/cir

Quick Review Sequence Block Metadata Title Academic Level Timing: Dates / Duration Expectations Events Duration Instructional Methods Assessment Methods Resources Nested Sequence Blocks Integration Blocks Valerie www.aamc.org/cir

Documenting clerkships Terri

Clerkship ‘Iteration’ Should represent core activities that majority of students are exposed to Select one rotation for upload A specific rotation First, Last Create an upload version of a rotation Upload all iterations – reporting issues Select one student’s schedule May be rotation or integrated May cross Academic Level Create Sequence Block in each Academic Level Terri

Clerkships What to Document Clerkship (Sequence Block) name All activities Didactics Conferences (M&M, Tumor Board, X-Ray, etc.) Rounds (Ward, Attending, etc.) In-patient and Out-patient Activities Assessment(s) (for each event and/or for Clerkship) How to document Clerkships are Sequence Blocks that are tagged as either integrated or rotation What if the clerkship is both integrated and a rotation? Choose integrated: Sequence Block dates compared to duration shows that clerkship is both integrated and a rotation Terri www.aamc.org/cir

Clerkship ‘Model’ Example Document ‘scheduled’ activities (e.g, 10-week clerkship) Didactics (Individual Events; 10 hours) Conferences (7 am, Noon, M&M, X-ray, Tumor Board, etc. as Individual Events or total, e.g,. 10 hours each; Total 60 hours) Grand Rounds (Individual Events or Total; 20 hours) Ward Rounds (Resident Rounds, Attending Rounds, Daily Rounds; Individual Events or Total; 50 hours) Total documented hours (140 hours) Total hours for clerkship (500 hours) Divide 360 hours using ambulatory/in-patient ratio 70/30 Ratio = 252 hours ambulatory; 108 hours in-patient documented as 252 hours Clinical Experience, Outpatient; 108 hours Clinical Experience, Inpatient

Clerkship Model Metadata Didactics (Slides are often included in Syllabus; first slide often includes Expectations/Objectives) Conferences (7 am, Noon, M&M, X-ray, Tumor Board, etc. – Expectations usually included in Syllabus) Grand Rounds (Often advertised as CME with Expectations listed OR pull from slides – good use of Nested Sequence Block) Ward Rounds (Resident Rounds, Attending Rounds, Daily Rounds – Expectations usually included in Syllabus) Clinical Experience Outpatient / Inpatient (use Student Clinical Experience Requirement Documentation from Syllabus; e.g., abdominal pain, headache, weakness)

Single Student Example All activities pulled from Evaluation Module or Student Schedule Evaluation Module should include Expectations Clinical Experience Outpatient / Inpatient (use Student Clinical Experience Requirement Documentation from Logbook; e.g., abdominal pain, headache, weakness)

Sequence Block Nesting A E B F I J 1 2 Sequence blocks can be nested No restriction on the number of nest levels Can represent a track or options where only some of the clerkships or courses are required Sequence blocks may be: Required, Optional, Required In Track Ordered, Unordered, Parallel You may require a minimum or maximum # of nested sequence blocks (selectives) A E B F I J 1 2 A E B F I J 1 2 Valerie A E B F I J 1 2

Clerkships You have 9 Academic Level 2 clerkships; 5 are required. 8 must be taken. They may occur in any order. How would we represent that? Clerkships, required = Required, order = unordered, minimum = 8, maximum = 8 Medicine, required = Required Duration – 60 days Surgery, required = Required Duration – 40 days Pediatrics, required = Required Duration – 30 days OB/Gyn, required = Required Duration – 30 days Valerie Neurology, required = Required Duration – 20 days Psychiatry, required = Optional Duration – 20 days Ambulatory, required = Optional Duration – 20 days Anesthesiology, required = Optional Duration – 20 days Emergency medicine, required = Optional Duration – 20 days

Documenting integrated and longitudinal curricula Terri

Integrated / Longitudinal Clerkships The Consortium of Longitudinal Integrated Clerkships (CLIC) states that longitudinal integrated clerkships (LICs) have the following common core elements: Medical students participate in the comprehensive care of patients over time Medical students have continuing learning relationships with these patients’ clinicians Medical students meet, through these experiences, the majority of the year's core clinical competencies across multiple disciplines simultaneously. Terri

Integrated / Longitudinal Clerkships Multiple options for documenting: Show as single Sequence Block Clerkship Model: Integrated Nested Sequence Block(s) Data Requirements/Sources: Same as for discipline-specific clerkships Issues: All / most students participate? Separate assessments? Terri

Documenting Electives Terri

What data should be entered? Best case scenario: Elective Course (Sequence Block) documentation should include: All Elective Courses: Name Instructional Method(s) Assessment Method(s) Objectives (Expectations) Terri

Electives What to Document Electives Program Requirements Use Timing_Dates to show entire time period when electives are offered Use Expectations to describe expectations of the Overall Electives Program Elective Courses Use Timing_Duration to show number of days for each Elective (5 days = 1 Week) Use Expectations to describe expectations of the Elective Course Events Each Elective Course must have at least one event (Instructional Methods, Assessment Methods, Resources, Expectations, etc.) Terri www.aamc.org/cir

Electives Options for Uploading Best Practice Other Options Electives Program Requirements as Sequence Block Elective Courses as Nested Sequence Blocks Other Options Elective Courses as individual Sequence Blocks Electives Program Requirements Only Use Timing_Dates to show Start and End Dates for entire time period when electives are offered Use Expectations to describe expectations of the Electives Program Use Description to describe specific requirements (Sub-I, x number of weeks per specialty, etc.) Terri www.aamc.org/cir

Electives Data Sources Curriculum Management Systems Scheduled activities or ‘Upload Version’ Online Electives Catalogs Duration – if multiple options are available, choose the most common Evaluation Systems Entering data Conversation between curriculum office and vendor / developer to choose best option Enter as ‘Representative Data’ Pull one iteration of each scheduled Elective Other ideas? Terri www.aamc.org/cir

Selectives What to Document Selectives Course (Sequence Block) Requirements Timing_Dates of the Selective Requirement Expectations of the Selective Requirement Selective Courses (Nested Sequence Blocks) Timing_Duration to show number of days for Selective Courses (5 Days = 1 Week) Expectations of the Selective Courses Events Each Sequence Block must have at least one event (Instructional Methods, Assessment Methods, Resources) Valerie www.aamc.org/cir

Selectives Options for Uploading (Best Practice) Selectives Course Requirements as Sequence Block Use Timing_Dates to show entire time period when Selective Requirement occurs Use Expectations to describe expectations of the overall Selectives Course Selective Courses as Nested Sequence Blocks Use Timing_Duration to show number of days for each Selective Course (5 days = 1 week) Use Expectations to describe expectations of the specific Selective Course Events Each Sequence Block must have at least one Event (Instructional Methods, Assessment Methods, Resources) Selectives www.aamc.org/cir

Selectives You have 8 fourth year sub internships; 1 is required. How would we represent that? Sub Internships, required = Required, order = parallel, minimum = 1, maximum = 1 Medicine Sub-I, required = Optional Duration – 20 days Surgery Sub-I, required = Optional Duration – 20 days Pediatrics Sub-I, required = Optional Duration – 20 days OB/Gyn Sub-I, required = Optional Duration – 20 days After previous Emergency Medicine Sub-I, required = Optional Duration – 20 days Family Medicine Sub-I, required = Optional Duration – 20 days Psychiatry Sub-I, required = Optional Duration – 20 days Anesthesiology Sub-I, required = Optional Duration – 20 days

Colleges,’ ‘Academies,’ ‘Bootcamps’ What to document Sequence Block for Requirement Timing_Dates (Entire time period) Expectations Nested Sequence Blocks for each Option Timing_Duration (5 Days = 1 Week) Expectations? Nested Sequence Block for each ‘Course’ in the Option Events Instructional Methods Assessment Methods Resources Valerie www.aamc.org/cir

Colleges,’ ‘Academies,’ ‘Bootcamps’ Sequence block, required=Required, order=Parallel, minimum=1, maximum=1 College 1 Required=Optional College 2 Required=Optional College 3 Required=Optional College 4 Required=Optional

Typical hrs/wk formal instruct** Multiple Sites What to Document One curriculum per institution LCME: “Show sum of lectures, conferences, and teaching rounds; range of hours if there is significant variation across sites” *Include the number of sites used for inpatient teaching and the number of sites used for outpatient teaching in the clerkship in the following format: # inpatient / # outpatient **Sum of lectures, conferences, and teaching rounds; show the range of hours if there is significant variation across sites   Clerkship Total wks % Amb. # Sites used* Typical hrs/wk formal instruct** Clinical encounter criteria† (Y/N) Patient log Terri www.aamc.org/cir

Multiple Sites Options for Uploading One iteration of each Sequence Block Representative curriculum that most students experience Issues? Terri www.aamc.org/cir

Tracks What to Document One curriculum per institution Options for Uploading Nested Sequence Blocks Sequence Block for each Academic Level Parallel Nested Sequence Blocks for each Track Additional Nested Sequence Blocks to enter individual courses for each Track Representative curriculum that most students experience Issues? Terri www.aamc.org/cir

Questions? Terri

Examples from medical schools Our thanks to the following schools for sharing their best practices: Susan Masters, UCSF Robin Rivest, Oakland Cindy Lybrand, ETSU Matthew Gerber, UCF Please share your ideas by sending 1-2 slides to: medaps@aamc.org. We’ll create a resources section on www.aamc.org.

UCSF Documentation of the Clinical Core & Advanced Studies (Susan Masters, Curriculum Dean) We document the core curriculum that all students take in the third and fourth years (Clinical Core and Advanced Studies, respectively) at a minimum at the level of clerkship objectives that are linked to graduation competencies. We layer longitudinal classroom and clinical experiences for all, and longitudinal clerkship programs that are for some students on top of the eight core clerkship objectives that all students complete either in block clerkships or in longitudinal programs. We include the one classroom-based required fourth-year course. We DO NOT document third- and fourth- year electives.

UCSF Documentation Map Clinical Core – Third Year Intersessions (Classroom Based, required for all) Longitudinal Clinical Experience (LCE, required for all) PISCES (Year-Long LIC, subset of students) KLIC (Year-Long LIC, subset of students) LIFE (Six-Month LIC, subset of students) Model SFGH (Six-Month LIC, subset of students) VALOR (Six-Month Clerkship Program, subset of students) Anesthesia FCM Medicine Neuro Peds Psych Ob/Gyn Surgery Advanced Studies – Fourth Year Coda (Classroom Based, required for all)

UCSF Mapping Linkage Strategy Graduation competencies are mapped to course/clerkship objectives. Only one graduation competency can be linked to a course/clerkship objective. Session objectives are linked to course/clerkship objectives. Session objectives can be linked to multiple course/clerkship objectives. All mapping elements can be tagged with UCSF Subjects (as a defined list) and with MeSH terms.

UCSF Mapping Flow Chart Graduation Competency Course/Clerkship Objective Session Objective Course/Clerkship Objective Graduation Competency Session Objective Course/Clerkship Objective Session Objective Session Objective Tags: Topics, MeSH Terms

UCSF Subjects Sixty-nine subjects derived from (1) a review of LCME standards; (2) the AAMC Graduation Questionnaire; and (3) USMLE reporting formats. Acute Care Clinical Reasoning Embryology Global Health Issues Immunology Pain Management Professional Development Ambulatory Care Clinical Skills Emergency Medicine Health Care Financing Internal Medicine Palliative Care Psychiatry Anatomy Community Health End-of-Life Care Health Care Quality Improvement Jurisprudence, Medical Pathology Public Health Anesthesiology Complementary/Alternative Health Epidemiology Health Care Systems Microbiology Pediatrics Radiology Behavioral Science Continuing Care Ethics, Medicine Health Informatics Neurology Pharmacology Societal Problems Biochemistry Critical Care Evidence-Based Medicine Histology Neuroscience Physical Diagnosis Socioeconomics, Medical Biostatistics Cultural Competence Family Medicine Human Development/Life Cycle Nutrition Physiology Substance Abuse Cell Biology Determinants of Health Gender and Cultural Bias Human Sexual/Gender Development Obstetrics and Gynecology Population-Based Medicine Surgery Chronic Care Diagnosis of Disease Genetics Human Sexuality/Sexual Function Occupational Health/Medicine Prevention/Health Maintenance Surgical Specialties Clinical and Translational Research Domestic Violence/Abuse Geriatrics Humanities, Medical Organ System Pathophysiology Problem Solving

OUWB Clerkships – What Data Academic Level 3 9 Clerkships 0 Integrated; all Rotations; Rotations weeks vary 1 Selective Course Academic Level 4 5 Clerkships 4 Sub-Internships 160+ Electives – not going into CI 1 Rotation for each (representative) Not including Electives at this time

OUWB – Clerkships – What Data Title, Dates, Timing, Academic Level, Expectations, & Events Expectations coded as K, S, A and mapped to Institutional Objectives and to PCRS Events – 2 ways On Student Calendar - Didactics, Conferences, Lectures Bulk Events (to show hours/occurrence, not on calendar) – Rounds, Inpatient/Ambulatory Includes: Instructional and Assessment Methods, Keywords, Objectives w/ Mapping, & Resources Curriculum Keyword List – 350+ terms by committee, includes Hot Topics

OUWB Clerkships – Data Sources Syllabi Moodle Page (LMS) Student Calendar/Schedule Clerkship Coordinators Clerkship Directors

ETSU Quillen College of Medicine Data Used to Document Clerkships M3 level = eight (six week) clerkships [core + community + specialties] Objectives mapped to Institutional and PCRS Event Title Event Duration Keywords (baseline from faculty (+ quality improvement is ongoing see next slide) Instructional Method(s) Assessment Method(s) Documented in Curriculum Management System Using a model within the New Innovation system (Generalist track / core) Subset of students complete an equivalent within Rural Primary Care Track (two rural sites)

ETSU Quillen College of Medicine Mapping expectations and keywords to clerkship events Running dual systems as part of migration efforts Combination of multiple sources (faculty input, syllabi) Documenting multiple iterations Use of model for core requirements Multiple sites have same / common objectives to achieve Keyword Lists: Institutional Education Objectives [PCRS] Institutional Threads / Cross-cutting Themes [can be case threads] National Standards and/or models for curricular content [Specialty / Field / Core Content] Reporting Topics LCME (Elements + DCI + Questionnaires) AAMC Reporting USMLE Content Outline / Categories [two / three levels] Monitoring curriculum committee changes or reviews related to topic / thread + EPAs (post BETA)

UCF College of Medicine Clerkships and Electives We are documenting, and submitting to the CI: Clerkship/Elective Requirements Clerkship Objectives Hours of Instruction in Clerkships and Electives Most data are pulled from official documentation We map clerkship-level objectives by pulling them from syllabi No attempt to map objectives at the event level We don’t have reliable access to the objectives for most of the electives No attempt to represent individual clerkship iterations as such Nested sequence blocks used to describe (most of) the orders and combinations in which students can take clerkships and electives These sequence blocks are the only M3/M4 periods given dates – clerkships, electives and their events are not tracked on the real-world calendar Each clerkship is given the expected number of hours of each type of instruction and assessment by “fiat”; for electives this is notional Keyword lists are taken from MedBiquitous or from LCME standards

Questions? Please use Q & A option to submit questions.

2015-2016 Schedule Staging available now School Portal Opens: August 1 School Portal Closes: September 30 Status e-mail sent to CI Primary Admin, Curriculum Dean, and School Dean: September 30 Data clean-up/reporting starts: November 1

2015-2016 Initiatives Gateway Exams Release School Name CEPAER Documentation

Best Practice for ‘Gateway Exams’ Major Exams for Promotion or Graduation Often multi-day exams Not related to a particular course or clerkship Often a series of unqueued cases Examples: Senior Clinical Competency Exam; CCX; Post-Clerkship Exam

Best Practice for ‘Gateway Exams’ Documenting Gateway Exams: Separate Sequence Block Duration expected for average student Include all Assessment Methods as Single Event (e.g., All Cases) or Split into Multiple Events (e.g., Case 1, Case 2.) No Instructional Methods Include Keywords for Content if possible Reporting is post-exam and aggregate If exam is split into multiple Academic Levels, reference the AL where most of the exam occurs Use Sequence Block Description to provide innovations, information such as year exam was implemented, etc.

Using Program Description to Designate Approval of School Name linked to CI Reports A Brief History CurrMIT allowed schools to run reports to find schools offering particular content or using particular instructional or assessment methods or resources Received both positively and negatively To expedite implementation of CI and ensure highest participation rate, schools were told all reports would be aggregate Received both positively and negatively  Many schools, as well as medical education researchers and media, want to be able to contact schools to learn more about what and how they are teaching certain content. From a school standpoint, including school names creates a collegial environment where schools can learn from each other. Our current process for meeting this need is a very manual process that includes contacting schools and asking them to contact the school, researcher, or media.

Using Program Description to Designate Approval of School Name linked to CI Reports Solution: Allow schools to choose whether they wish to have their name available for benchmarking and research School name will be in a separate data file – no link to actual data in report Schools Included in This Report link will include: School Name, Curriculum Dean Contact Information, Primary CI Contact Information

Using Program Description to Designate Approval of School Name linked to CI Reports Implementation August 1, 2016, for 2015-2016 Data: Use Program Description to include the text: ReleaseName=Y OR ReleaseName=N No ReleaseName text will be the same as ReleaseName=N This should not require changes to programs – simply adding text to a field For systems that do not have this option, an e-mail can be sent to ci@aamc.org. E-mail will need to be sent each year.

Best Practices for CEPAER Documentation CEPAER: Core EPAs for Entering Residency 13 EPAs Linked to PCRS Competencies In Pilot Phase (10 schools) Not a requirement, but being documented at many schools Document as CEPAER-01 – CEPAER-13 Detailed information (included PCRS mapping) can be found at: https://www.aamc.org/initiatives/coreepas/

Questions? List of upcoming Curriculum Inventory Webinars and materials from today’s session: www.aamc.org/cir