Curriculum Inventory Webinar August 8, 2018 Angela D. Blood Walter Fitzwilliam Michael Cameron John Nash Whitney Staiger

Agenda Medical School Highlight: Samara Ginzburg, Hofstra University presents “Self-Directed Learning” Verification Report improvements Sample of Verification Reports (old and new) on CI website NEW Verification Report Guide on CI website Preparation for CI season 2017-18 Project update: Structured survey needs assessment CI @ Learn Serve Lead Next meeting: Wednesday, September 12, 1 pm ET

Self-Directed Learning Samara Ginzburg, MD Associate Dean Case-based Learning Zucker School of Medicine AAMC Curriculum Inventory Webinar August 2018

Goals for Presentation Understand LCME element on SDL/Lifelong Learning Understand approach to SDL /Lifelong Learning as a unified sequence at Zucker SOM Discuss mapping challenges related to SDL Explore questions, challenges & suggestions for solutions from participants

Traditional Medical Education

Self-Directed & Lifelong Learning “Tell me and I forget, teach me and I may remember, involve me and I learn.” -Benjamin Franklin

LCME Element 6.3 SELF-DIRECTED AND LIFE-LONG LEARNING “The faculty of a medical school ensure that the medical curriculum includes self-directed learning experiences and time for independent study to allow medical students to develop the skills of lifelong learning.” Self-directed Learning Involves: Medical students’ self-assessment of learning needs Independent identification, analysis, and synthesis of relevant information Appraisal of the credibility of information sources

LCME Element 6.3 NARRATIVE RESPONSE Describe the learning activities, and the courses in which these learning activities occur during the first two years (phases) of the curriculum, where students engage in all of the following components of self-directed learning as a unified sequence: Receive feedback on their information-seeking skills Assess the credibility of information sources Share the information with their peers and supervisors Synthesize information relevant to their learning needs Analyze information relevant to their learning needs Identify information relevant to their learning needs

PEARLS=Patient-Centered Explorations in Active Reasoning, Learning & Synthesis

Curriculum Overview: First 100 Weeks

PEARLS: Foundation of Curriculum

PEARLS: Identify & Analyze What do I know? What don’t I know What do I need to know?

PEARLS: Synthesizing & Sharing Information Lifelong Learning Higher-order Questions Based Upon LO's Students Present Information to Peers Facilitate Discussion Reconcile Information No Notes

PEARLS: Sharing Information, Challenging Sources & Feedback

PEARLS Wrap-up: Feedback on Information Seeking Skills “My research somewhat failed me today. There was a major point discussed in group that I did not come across in my research. I need to go back and see if it was in my sources and I missed it or if I need to add another resource for future use.” “I feel unsettled after our discussion about ___ because there was so much conflicting information based upon the different resources we all used.” “I really liked the chart you shared in group today-it helped organize the material clearly. I am definitely going to use that book next time.”

PEARLS: Analysis of Ongoing Learning Needs Sharing Expectations Suggested Learning Objectives & Answers to Weekly Essays Released Student Identification Of Learning Needs Students Compare & Contrast Identify Outstanding Learning Needs Filling the Gap Course Directors Address Outstanding Issues at Review and Reinforcement Sessions

PEARLS: Complete all Components of SDL Receive feedback on their information-seeking skills Assess the credibility of information sources Share the information with their peers and supervisors Synthesize information relevant to their learning needs Analyze information relevant to their learning needs Identify information relevant to their learning needs

Reporting of Self-Directed Learning Source: AAMC Curriculum Inventory, 2016-2017 Sum of SDL Events as Primary Instructional Method at Each School Sum of SDL Events as Non-primary Instructional Method at Each School

Mapping of Self-Directed Learning Curriculum Inventory: Instructional Methods LCME Standard 6: Competencies, Curricular Objectives and Design- Instructional Formats CBL: patient cases stimulate discussion Discussion, small: exchange opinions/ideas Independent Learning: instructor guided; outside class protected time PBL: cases, problem solving, SDL, team skills SDL: learner initiated, dx, form goals, identify resources, evaluate outcomes Lecture Lab Small group (includes case-based or problem-solving sessions) Patient contact Other Challenges: May not capture full breadth; especially for mapping examples from Element 6.3 on SDL Challenges: Not mutually exclusive

Mapping Self-Directed Learning Examples of other challenges mapping SDL? Ideas for solutions and ways the Curriculum Inventory can help?

Goals for Presentation Understand LCME element on SDL/Lifelong Learning Understand approach to SDL /Lifelong Learning as a unified sequence at Zucker SOM Discuss mapping challenges related to SDL Explore questions, challenges & suggestions for solutions from participants

feel free to reach out with qUESTIONS Samara Ginzburg, M. D feel free to reach out with qUESTIONS Samara Ginzburg, M.D. Associate Dean for Case-based Learning Zucker School of Medicine at Hofstra/Northwell samara.ginzburg@hofstra.edu 516-463-7501

Verification Report improvements for 2017-18 Readability: we changed the font, increased the font size, and tried to make better use of white space Explanations: we added guide information for every section of the report, explaining what data is and is not included in given tables, how amounts are calculated, etc.  Wording: we added clarity to the glossary, retitled tables, retitled columns, and added columns within tables to more accurately represent which data is provided OLD NEW I know a guide was discussed and promised last year. We will have it ready for this summer’s upload. I began with a write up of what I thought were all the pressure points of the Verification Report. We asked for feedback in one of our monthly webinars a couple of months ago. I presented the list of challenges to the AAMC development team, and presented the planned changes with opportunity for feedback to the CI Advisory Board and CICA groups in June 2018. Some of the challenges with the Verification Report will take more time to execute than is possible to do well in just a couple of months, so the improvements you see here are the ones that were able to be accomplished with a very aggressive timeline. If you have more changes you’d like to see please tell us; we are actively working on improvements for 2019, and given the timeline, we can think more radically for 2019. As an example, for some schools the report can be very long. If you wanted to distribute this to your various faculty, we recognize that it would be easier for you if the data weren’t collated into one giant table (e.g., the Session Event Details table at the end of the report). This was a change that we could consider for 2019.

OLD NEW All tables are now numbered Underlines within tables will be clickable links so you can jump to other parts of the PDF. Trying to include links which go back to our website, but this way you won’t have to dig to find connecting documents. We may not be able to make all web links within the document single word clickable hyperlinks, because not everyone will have PDF readers which can do that. We may have to leave the full web addresses in the PDF so that everyone can click on them. Trying to be explicit about which tables most align with LCME DCI. Table formats may not match exactly, e.g., “typical” hours per week Relabeling tables. It might have said “all” but it really was primary. We’re investigating how to move beyond primary in reporting for 2019. Notice the “included in grade” where it spans, added narrative assessment to align with recent DCI changes. Formative is missing here but it wasn’t dropped, this is part of the ongoing adjustments, this isn’t the finished version. Trying to be explicit about how formulas are calculated, what data makes it into the data.

Verification Report Resources

Preparation for 2017-18 upload Portal open August 1 – September 30, 2018 Staging site CI submission portal CI Portal User Guide CI Newsletter July 2018 Staging: Contact ci@aamc.org for gaining access to your institution’s staging account information On the CI portal: Check your institution’s roles (Primary CI Admin, Dean, etc.) and assigned sender MANY more resources on the CI website

Data in upload Documentation of EPAs not required but accepted if submitted Resources not required by strongly encouraged If you submit resources, must use ID codes just as you would for instructional or assessment methods Spellcheck your submissions

Quality assurance of submissions: Validations Medbiquitous Specifications, AAMC Business Rules

Quality assurance of submissions: Validations After a successful submission, each school’s data is processed through additional validations; examples of what we look for: 4000 character expectations/learning objectives Duplication of expectations/learning objectives across sequence blocks/courses Academic levels over 600 days long Having no assessment events or only assessment events Relating more than 11 PCRS to a single Objective Identical events

System/AAMC Business Rule changes for 2017-18 Competency object titles must contain letters/words, not just numbers Sequence Blocks start and end within 8 years of the Reporting Start and End Dates. Start and End dates for Sequence Block Events are within 8 years of Reporting Start and End Dates. Resources are referenced by <ResourceType> and will be in the same style as Instructional and Assessment methods RE0##, e.g. RE004, RE013 Other text will result in a business rule violation

Project Update: Structured Survey Needs Assessment CI Committee members identified Jeanne Farnan Colleen Grochowski Tanya Horsley Brian Mavis Arianne Teherani Julie Youm Draft of survey in review I presented details in our last webinar about a proposed 3 year plan and needs assessment underway, wanted to provide an update for you now.

CI @ Learn Serve Lead 11/2 AM: Curriculum Inventory - Open Business Meeting (Open) Plans for future CI initiatives 11/2 PM: Curriculum Inventory – Closed Business Meeting and Reception (Closed) CI Advisory Board, CI Research Group, CICA, and Standardized Vocabulary groups 11/4 PM: AAMC Curriculum Inventory: 2017-18 Knowledge Sharing (Open) Preliminary results of 2017-18 As a participating school, you have access to FREE customized reports. If you have Learn Serve Lead presentations for which you’d like to include CI data, please send your request by September 15: https://www.aamc.org/data/479586/requestaamcdata.html Search existing reports here: https://www.aamc.org/initiatives/cir/curriculumreports/ The online portal is still being updated, but these dates/times are true, and posted on the CI homepage as you requested last month

Next meeting: Wednesday, September 12, 1 pm ET Registration Links posted on the Resources to the Establish Your CI section of the CI website: https://www.aamc.org/initiatives/cir/423212/ciadmingroupwebinars.html Please send agenda items to ci@aamc.org