Integrating Best Practices of Participant Evaluation Clinical Instructor Intensive Adrienne Small, DNP, FNP-C, CNE, CHSE Medical Instructor Duke University School of Nursing

Standards of Best Practice For Participant Evaluation Reference: INACSL Standards of Best Practice: SimulationSM https://www.inacsl.org/i4a/pages/index.cfm?pageid=3407

All simulation-based experiences require participant evaluation Determine the type of evaluation prior to the simulation experience: - Formative evaluation - Summative evaluation - High-stakes evaluation 

Participant evaluation is: Directed by desired objectives and outcomes  Guided by the type of evaluation:  formative, summative, or high-stakes

Formative Assessment •Learner-centered •Ongoing, focused on learning and growth •Provides feedback to learners •Measures progress toward outcomes •Can be ungraded

Methods of Formative Assessment

Know the difference between a pattern and an isolated incident Know what to look for Observations should be recorded so that you are able to give students clear examples Know the difference between a pattern and an isolated incident Instructor's biases and values Staff or preceptor feedback Simulations:  feedback from lab faculty or SPs

Observation

Questioning

SUMMATIVE EVALUATION -Graded -Focused on competencies and outcomes - May occur at the end of a unit or module - End-of-course or end-of-program testing *Students should be told what to expect * For instructors:  end-of-course evaluations SUMMATIVE EVALUATION

Tools for Summative Evaluation: •Anecdotal notes •Clinical evaluation tools •Written work •Written exams •Practical exams •OSCEs

Clinical Evaluation Tools Objective assessment of clinical abilities and professionalism Can be formative (e.g. Midterm eval) or summative (e.g. Final eval) Specific to programs/program outcomes Evaluate competencies and behaviors Include student self-eval and faculty eval Increased complexity and autonomy are expected as students progress

Types of simulations that can be used for evaluation Simple:  Task trainer with a skilled observer Mechanical:  hi-fidelity mannikin with a skilled observer Standardized patients Virtual reality: programs like I-Human and Shadow Health

Objective Structured Clinical Examination An instructor observes the students perform a certain task, or an examination. Standardized patients are often used for OSCEs. OSCE

High Stakes Testing – benefits for students Conducted at the end of a learning process  Can also be used at other times to assess gaps in knowledge Identifies safety issues in learning. Provides proof of learners' clinical competencies Assesses readiness for real-world practice

High Stakes Testing – Caveats for instructors MUST be pilot tested to work out bugs Consequences and outcomes must be explained to participants Participants should have been exposed to simulation in the past Needs a comprehensive rubric or checklist that clear outlines expectations Train the raters on the tool Establish interrater reliability Video recording is preferable Use more than 1 evaluator for each participant, either live or recorded