Development of a Video Based Tool for Resident Teaching and ACGME Competency Assessment STFM Workshop - April 2006 Robin Helm, MD, Theresa Drewniak, PhD, Nancy Havas, MD, Suzanne Gehl, MD, Deb Simpson, PhD, Department of Family and Community Medicine

Objectives Demonstrate video-based tools used to teach and evaluate the ACGME competencies Discuss process of developing the video-based tools Describe and pilot a process for developing of own video-based case scenarios

Interdisciplinary Project Family Medicine: Theresa Drewniak, PhD; Suzanne Gehl, MD; Nancy Havas, MD; Robin Helm, MD General Internal Medicine: Steven Denson, MD; Julie Mitchell, MD; Monica Ziebert, DDS, MD Geriatric Medicine: Kathryn Denson, MD Edmund Duthie, MD; Diana Kerwin, MD Geriatric Psychiatry: Harold Harsch, MD Medical Education: Thomasine Dankert; Diane Brown; Dawn Bragg, PhD; Mary Heffron, PhD; Deborah Simpson, PhD

Needs Assessment Tools developed in response to: Data from WI-based PC residency faculty Literature review identifying gaps Competencies Primary care knowledge and skills in geriatrics

ACGME Competencies Patient Care Medical Knowledge Practice-Based Learning and Improvement Interpersonal and Communication Skills Professionalism Systems-Based Practice

ACGME Competencies Patient Care Medical Knowledge Practice-Based Learning and Improvement Interpersonal and Communication Skills Professionalism Systems-Based Practice

Tool “Table of Contents” (Level of importance 1 to 5 1=highest)

Tool Components Instructor Guide Resident Assessment Sheet Video Trigger Score Key

Instructor Guide Background > 2 Competencies Systems-based Practice-based Interper/Comm Skills Professionalism Specific Objectives Instructions and time 10-12 minutes to complete 10 minutes to score

Video

Resident Assessment Sheet and Score Key

OSVE DEVELOPMENT Spiral Process

Development Idea → Clearly identified purpose/focus Expertise What goes into Developing an OSVE tool? Idea → Clearly identified purpose/focus Concept/approach @ resident/faculty struggle Core Curriculum/ACGME gap Expertise Education (objectives, assessment) Subject matter (medical errors, differences between sites) Videography Project Management (meetings, editing, files) Persistence & Patience

Project Management

Script Folder

Identifying your Resources Information Resources Needs Assessment to ID topic focus Subject matter - best evidence Setting Fidelity but controlled (background noise) Creativity (pager = bull horn) Budget (In-Kind vs $) Faculty Authors, Educationalists Production Actors, props Videographer/editor Pre- Post Production Duplication of scenes, tools, edit Dissemination / distribution (e.g., CD, Web)

Developing Your Own OSVE Identifying a Competency Assessed Needs (Faculty/ Residents) Medical Knowledge vs. Others Selecting a Subject Familiar scenarios Draw upon competency goals Physical / Budget restrictions

Identifying Resources Information Resources Evidence Based Expert opinion (collaborators) Applicable to 5 years from now Setting Use what you have

Groups Form groups (3 to 4 in each group) Introductions within group Work outlines provided ACGME Competencies provided

Work Outline Step 1: Select a topic Step 2: Develop scenario summary Step 3: Identify competencies Step 4: Script scenario Step 5: Develop 3-4 question/answers Step 6: Assign points to questions Step 7: Fill out the Instructor’s Guide

First Drafts Draft an outline for a first draft script 50 minutes (20 minutes+ 30 minute break) Work on your script for review and discussion following the break Refreshments are available in hall

BREAK Take a Break! Refreshments are available in hall Work on your script for review and discussion following the break Choose a spokesperson for your group

Debriefing of Script Development Each group provide brief synopsis of scenarios and scripts What was easiest? What was most challenging? Were any of the ACGME competencies easier or more difficult than others to script? Create questions and answers for?

Piloting Assessment Tools Tips for Piloting Assessment Tools Noon Conference Morning Report Have other faculty give instructions Faculty to complete evaluation of tool

Application of Video Based Tools Conferences Integrate into lectures Individual teaching/assessment Patient rounds Nursing home

Discussion and Questions

Conclusion Video-based tools can be used to teach and evaluate the ACGME competencies Several set process exists for developing the video-based tools These video-based tools can enhance training in your residency

Thanks for your Participation! To find out more about creating your own OSVE, contact: rhelm@mcw.edu tdrewniak@mcw.edu nhavas@mcw.edu sgehl@mcw.edu dsimpson@mcw.edu