1. Learning and Applying Concepts while Having Fun: A Perfect Training Environment Learning and Applying Concepts while Having Fun: A Perfect Training Environment Karen Szauter, MD and Ann Frye, PhD University of Texas Medical Branch, Galveston TX Introduction The principles of educational measurement are neither easy to understand or simple to teach A foundational understanding of validity, reliability and measurement error is essential for all medical educators To master the basics, both explanation and application of these principles is needed. Over the past several years, we have worked with a MedEdPORTAL resource to guide the delivery of workshops for faculty and professional colleagues addressing these challenging concepts. Workshop Sequence Brief didactic addressing the concepts of validity, reliability and measurement error Interactive focus: Identification of the “best” chocolate Attendees brainstorm on characteristics of chocolate which can be used in grading Group consensus: 5 characteristics are chosen Small groups each take one characteristic and develop anchors for a seven point rating scale Faculty Development: a spokesperson from each group describes the intended use of the scale; all five scales are presented and combined into a single rating form Individuals apply the newly developed rating scale to multiple samples of chocolate. Categorical ratings and an “overall” score are applied Scores are collected and the score distributions are shared with the participants Applications of the concepts of validity, reliability and measurement error are addressed in a debriefing. Publication Educational Measurement Workshop: A Sweet Approach to Understanding the Basic Principles of Educational Measurement Simpson D, Meurer L. MedEdPORTAL; 2007 Application We have adapted this workshop for our Scholars in Education Program (institution-wide 20 month faculty development program; interdisciplinary) We have used it for our Clerkship Directors conference The workshop was also presented at the Association of Standardized Patient Educators annual meeting (2010 ) Outcomes The debriefing sessions have resulted in rich and insightful discussions. Post session written comments have included: “Had not thought of many of these aspects of rating” “…inherent bias in scales…” “…need to provide greater focus in my grading criteria…” Participants have successfully integrated the foundational concepts and have recognized the importance of application in their day to day work. Immediate feedback has included recognizing: the importance of piloting rating scales a need for attention to language used in scale anchors the critical need for faculty development prior to using a rating scale the potential for environmental influences on ratings Evaluations for this session have been uniformly positive. Summary This MedEdPORTAL resource is exceptionally well developed and easy to adapt in a variety of settings. We are indebted to Deb Simpson and colleagues for this creative and sweet approach to introducing important measurement concepts to our colleagues Eating chocolate as part of work is a plus.