Review of Year 1 HAE II course Course occurs in the winter term of Year 1 Course Director – Virginia Lyons, PhD Course has 67 curricular hours Course was last reviewed in Oct 2007 Subcommittee review team: Aniko Fejes-Toth, David Mullins, Will Boyce

Course Objectives – Content Review There are 16 course objectives that fulfill Geisel competencies as follows: 10 address specific knowledge in the preclinical domain, and seem appropriate 1 addresses clinical care skills 2 address communication skills 6 address components of professionalism 1 addresses personal improvement Currently no course objectives are mapped to Geisel competency 6 (health care systems), which is appropriate for this year 1 course.

Course Objectives – Content Review Course objectives are: provided in the syllabus written in the correct format appropriate for a year 1 course

Objectives: Step I Brochure The Human Anatomy and Embryology (HAE) course content was mapped to the NBME Step 1 Content Description and General Information Brochure Generally, the course covered NBME topics associated with embryology and anatomy and function of the nervous, musculoskeletal, respiratory, cardiovascular, gastrointestinal, renal/urinary, and reproductive systems (all anatomic categories on the Step 1 Content Outline) The course does covers a variety of diseases and injuries – For example, content related to the musculoskeletal system included herniated disks (back), rotator cuff tears (shoulder), and carpal tunnel syndrome (wrist) – Specific diseases and injuries were discussed for all major systems

Comparison to Discipline-based Organization In 1997, the AACBNS (Association of Anatomy, Cell Biology and Neurobiology Chairpersons) produced a document about curriculum content in the anatomical sciences in the US/Canada; this appears to be the most recent document of this type (there is a 2007 version for schools in Great Britain and Ireland) The content of the HAE course closely correlates with the recommendations in this document, covering all of the major learning objectives for human gross and developmental anatomy

Session Objectives Session objectives are provided to students at the beginning of each chapter section of the iBook Session objectives are written in the correct format Occasionally revisions had been made to the session objectives that were not updated in Ilios (so the session objectives in Ilios did not match perfectly to those in the iBook)

Unplanned redundancy The curriculum database was used to assess redundancy in the curriculum regarding various topics in the course For broad concepts (e.g. “altered anatomical structure,” “gross morphology”), there was no overlap with other courses For more specific concepts (e.g., “embryology”), limited redundancy (5 non HAE session objectives) was complementary (for example, in SBM201, a session objective is to “Describe the relationship between embryology and anatomy in the pathophysiology of congenital lung disease”) For specific topics, redundancy existed that was planned or a logical pedagogic progression (e.g. material introduced in HAE that was expanded in SBM courses), consistent with HAE serving as the introduction to most anatomic structures and systems

Summary regarding Objectives Course objectives are provided in the syllabus and are written in the correct format using verbs with measurable outcomes Course objectives correlate well with material in the Step I brochure and the standards of the national professional organization Session objectives are provided to students and written in the correct format Significant unplanned redundancy was not found

Course Learning Opportunities Large group interactive session 21 hrs. (31%) these are flipped classroom sessions where students work with their colleagues to solve problems Small group conferences 4 hrs. (6%) 16 students (divided into groups of 4) work together to solve clinical case problems, then discuss with the entire group and facilitator Laboratory 39 hrs. (58%) students work typically in groups of 4 doing anatomical dissection, osteology and radiology Clinical Correlation 3 hrs. (5%) clinical faculty discuss the otoscopic exam, craniofacial defects and demonstrate laryngoscopy Optional reviews before quizzes/final exam (6.5 hrs.)

Summary regarding Pedagogy Course uses a flipped-classroom model, thus the percentage of lectures is low (5%); the 3 lectures in the course are clinical correlations presented by clinical faculty Students have numerous opportunities during class time to practice applying their knowledge Course does not currently include opportunities for students to define their own learning objectives (part of the LCME definition of active learning) Optional reviews are provided to the entire class; struggling students are also offered faculty-led group tutoring

Assessment Written Quizzes (50% of course grade) – Typically cover one regional unit (e.g. thorax) – 20 questions on each quiz (MCQ, matching, fill-in-blank) Final Exam (40% of course grade) – written portion: 50 questions (MCQ, matching) – practical portion: 50 questions (fill-in-blank) Readiness tests (5% of course grade) - assess preparation for flipped classroom sessions Team performance (5% of course grade) - credit awarded for attending/participating in laboratory and small group sessions

Assessment - Quizzes What formats do they use for questions? – MCQ 67.5%, matching questions 20%, image recognition 12.5% – The questions correlate extremely well with session objectives. – ~70% of the questions are “recall” type; however, the nature of the material necessitates this. – ~50% of the questions employ patient-related stems. – Some questions require application of knowledge – No negative formats were used for quizzes

Assessment – Final Exam Written portion 44% of the questions were MCQs 36% were matching questions 20% “mixed questions”, matching structures with embryonic origin The questions correlated well with session objectives. Practical portion Consists of 50 identification questions Students identify structures on cadavers, bones, radiologic images and cadaver cross-sections

Assessment – Final Exam The questions were roughly proportional to the two main topics covered in this term (i.e. lower extremity and head/neck) ~80% of the questions on the written portion of the exam are “recall”; ~36% applied to a clinical situation. No negative formats were used for the final exam. All questions on the practical portion of the exam require students to identify structures; 16% require students to apply their knowledge to identify structures in images (e.g. xrays, CT scans, etc.)

Assessment The readiness tests are novel and assess the results of active learning Assessment of team performance (5% of course grade) is a good idea; encourages team-work.

Summary regarding Assessment Excellent correlation between questions and session objectives Appropriate balance of topics covered Format of the questions is well balanced, MCQ, matching & fill-in-the-blank. Due to the nature of the material, the majority of the questions are “recall” but “application” type questions are also present. Even in “recall” questions stems often are clinical/patient-related. Overall excellent job in assessment.

Measur es of Quality – AAMC GQ BASIC SCIENCES Geisel mean 2008 Geisel mean 2009 Geisel mean 2010 Geisel mean 2011 Geisel mean 2012 All schools means 2012 Behavioral Science Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology Immunology Microbiology Neuroscience On Doctoring Pathology Pathophysiology of Disease Pharmacology Physiology Measures of Quality – AAMC GQ “Indicate how well you think that instruction in anatomy/embryology prepared you for clinical clerkships and electives.” [1=poor; 2=fair; 3=good; 4=excellent]

Measures of Quality – Step I 2009*2010*2011*2012* Means TRADITIONAL CORE DISCIPLINES Biochemistry Biostatistics/Epidemiology Genetics Gross anatomy/Embryology Histology/Cell Biology Microbiology/Immunology Pathology Pharmacology Physiology *values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measur es of Quality – AAMC GQ Year 1 courses Overall Satisfaction AY Human Anatomy and Embryology II 4.57 Human Anatomy and Embryology I 4.33 Physiology-Renal 4.32 Biostatistics and Epidemiology 4.28 Metabolic Basis of Disease 4.23 Biochemical and Genetic Basis of Medicine 3.97 Neuroscience 3.90 Physiology-Endocrine 3.81 CTO 3.79 Basic Science of Microbial Disease 3.78 Virology 3.71 General Pathology 3.46 Physiology-Cardiovascular 3.45 Physiology-Respiration 3.38 Immunology 2.35 Measures of Quality – Course Reviews scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality – Course Reviews HAE II 2011 (23%)* HAE II 2012 (97%)* HAE II 2013 (88%)* Overall satisfaction of course Overall usefulness of lectures/large group not surveyed Overall usefulness of small groups not surveyed Overall usefulness of laboratories not surveyed Congruence of assessment questions to material emphasized in course Clarity of learning objectivesnot surveyed 4.52 *student participation rate on course evaluation

Measures of Quality – Course Reviews Strengths: Faculty “The faculty for anatomy are outstanding. They are extremely accessible and do a great job of organizing the material.” “The anatomy faculty is top notch and are some of the most caring, intelligent teachers I have ever had.”

Measures of Quality – Course Reviews Strengths: Organization/Pedagogy of the Course “The anatomy course is by far one of the best organized and delivered courses.” “The entire course was incredibly well organized. There were so many resources provided to the students, including the special resources on Blackboard, which were very helpful.” “I loved the "flipped classroom" sessions--always useful and engaging.” “The small group discussions were excellent in helping me work through difficult questions/concepts.”

Measures of Quality – Course Reviews Strengths: Course Materials “The course materials were excellent. Clearly written, in a logical presentation, and concise.” “I liked the anatomy website, especially radiology and visible human slides.” “The dissection videos Dr. Catlin made were really helpful in the course.” “The information presented in the ibooks was very well organized, very clear, and made learning the material as easy as possible.” “I loved the interactive activities in the ibook and online.”

Measures of Quality – Course Reviews Suggestions for Improvement: Embryology “I felt it would be beneficial to learn embryology after the cadaver lab and large group sessions for the specific portion of the body we are studying. I think trying to learn the embryology of a structure before really knowing where it is or what it does was a bit confusing at times and led me to not retain the embryology as well.” “It would also be nice to put embryology at the end so that we can draw connections to structures that we have previously learned to their embryologic origins. Learning embryology first was confusing because I did not know what or where the adult structures were.”

Measures of Quality – Course Reviews Suggestions for Improvement: Pedagogy “I would like to see more small group case discussion sessions. We had a considerable amount in the first term and only one or two maybe in this second term.” “  More small group sessions would be beneficial” “I would have liked to switch lab groups for the second term. I think it would have been a nice opportunity to work with different people.”

Measures of Quality – Course Reviews Suggestions for Improvement: iBooks “Images in the iBook that were not in the gallery were too small and there is no way to zoom in- even in the gallery some of the words in the images could not be read.” “Consistency of iBook formatting so that all images can be opened in gallery format (largest viewing size available).” “iBook videos could be improved…I think the videos would be better if they were more dynamic. ”

AAMC GQ data places the course in the “good-to- excellent” range, and students score above the mean on Step I in anatomy/embryology Students rate the course in the “very good to excellent” range; it is the highest rated course for Y1 Strengths in the course include dedicated faculty, clear organization, numerous course resources and engaging pedagogies Improvements could be made in the placement of embryology sessions, number of small group sessions and the format of the iBook Summary regarding Measures of Quality

Summary of Recommendations There are no significant problems with the course objectives; a few session objectives in Ilios need to be updated to match the objectives currently distributed in the course The percentage of lectures in the course (5%) is well under LCME guidelines, thus does not need to be adjusted The course director should explore opportunities to include activities in the course that meet the current LCME definition of active learning

Summary of Recommendations Assessment questions are well-written and correlate with objectives/material emphasized in the course; some improvement could be made in the number of questions that assess application of knowledge The course is well-taught and highly regarded. Minor improvements could be made in the embryology sessions and the newly created course iBooks.

Action Plan The course director will update any session objectives in Ilios and/or the new AAMC curriculum database to make sure they match the session objectives distributed in the course We will pilot a PBD activity this year (problem based dissection) that meets the LCME definition of active learning The course director will review questions on the quizzes and exams and incorporate more application questions where appropriate Embryology sessions for particular organ systems will be reordered in the schedule so that students learn the adult anatomy first (this is also being done in the fall term of the course) We are revising the formatting of the course iBooks to incorporate student suggestions