1. Nursing Simulation Evaluation Maria Overstreet, PhD (c), RN, CCNS

2. First Question to Ask To do or not to do…that is the question? Why evaluate? Evaluate what? What is the purpose of your simulation? Is purpose to practice, to learn, to experiment… Is purpose for competency?

3. After determination of purpose… Question becomes: –What am I to evaluate? Performance Knowledge Skill Attitude Confidence Teamwork Communication

4. Then…How will I evaluate? Tools to measure outcome We must develop the tools Scant few in nursing (see Jefferies, 2006) Have developed one…not tested for validity or reliability measures Have tested face and content validity Medical field has worked on some specific to discipline using simulation, anesthesia (ANTS Tool)…

5. Competency School of Nursing Student Competency –Evaluation of skill mastery –Evaluation of synthesis of knowledge –Evaluation of decision making under stressful condition Hospital Nurse Competency Licensing Competency License Renewal Competency Think about CPR renewals… Think about patient care standards… Think about legal issues…

6. Evaluation of Curriculum Where to add simulation in curriculum SON are going about this differently. Some use clinical hours (25% simulation) –Non-clinical days vs. clinical days Some are adding to lecture time –Pharmacology –Physiology Some are adding simulation to laboratory

7. Faculty It is the faculty who must decide where and when to use simulation to assist the student. We are currently looking at the simulation content to be concepts our students have the most difficulty grasping. Such as insulin administration, medication management, and concepts from specialty rotations (OB, Peds, Psych, M/S).

8. Example Review Insulin Administration Simulation Purpose Goal Objectives Plan Action Evaluation

9. Checklist of behaviors for student during diabetic medication error simulation Technical Skills: I/O measure urine I/O measure IV bag for beginning of shift Prepares insulin Administers insulin Prepares and administers other medications Maintains safe environment: bedrails Assessment: Brief neurological assessment for orientation, pupils, skin Assess heart Assess lungs Assess IV site Assess injection sites Communication: Requests nurse or instructor to verify insulin calculation Requests nurse or instructor to verify insulin preparation Talks with charge nurse, instructor, or physician about concerns with medications Talks with charge nurse, instructor, or physician as to when cystoscope is scheduled / NPO Rewords technical language for patient educational level Body language inviting for patient Uses professional body mechanics Critical Thinking: Reviews V/S from tech Reviews BG from tech and rechecks BG if value out of norm for client Questions to hold metformin order due to possible IV dye Questions order for HCTZ on dehydrated patient Does not try to teach patient who is slightly confused Organizes am duties within priority Question need for O2 @2LNC (confusion, sat 93%) versus infection cause of confusion Questions regular diet order for diabetic Questions order for activity with confusion, is the patient steady?, can he follow directions?

10. *Rater will evaluate the overall performance based on the decisions and interventions chosen by the student to implement. A rating scale of 0 – 5 will be used. Rating Scale : 0 = did not perform 1= poor performance 2= performed, needs improvement 3= acceptable performance 4= good performance 5= exceeds level of peers

11. Essentials for Student Performance during Simulation Experience © Overstreet/Miller February 3, 2008 (developed from “The Essentials of Baccalaureate Education for Professional Nursing Practice, American Association of Colleges of Nursing) Category I: Technical Skill Category I: Technical Skills Performance of interventions: vital signs, bed making, bath, recording I/O, basic and advanced skills including learner’s hand-eye coordination. 0 1 2 3 4 5 Completeness 0 1 2 3 4 5 Accuracy Comments:*Rater will evaluate the completeness of the skill and the accuracy of the performance

12. Category II: Assessment Focuses assessment depending on the diagnosis and responses from patient Completes all components of the assessment required by diagnosis and patient response 0 1 2 3 4 5 Chosen assessment items 0 1 2 3 4 5 Accuracy Comments:*Rater will evaluate evidence of assessment performed and the accuracy and preciseness of the assessment

13. Category III: Communication Performs therapeutic communication techniques when applicable Communicates effectively and clearly with patient/family/colleague Is or becomes aware of body language of patient. Uses own body language to communicate effectively 0 1 2 3 4 5 Communication techniques 0 1 2 3 4 5 Professionalism Comments*Rater will evaluate the appropriate use of communication techniques and/or body language in compassionate care of patient/family. Evaluate professional communication with colleagues

14. Category IV: Critical Thinking Reviews data on hand and asks questions for clarification when appropriate. Is able to make decisions at the patient’s bedside which will effect the quality of the patient/family care/life 0 1 2 3 4 5 Decisions and Interventions Chosen Comments:*Rater will evaluate the overall performance based on the decisions & interventions chosen to implement

15. Using 0-5 Rating Scale Run clip of student performing during medication simulation and have audience rate performance.

16. Audience Rating of Student Behaviors Technical Skill –Completeness 0-5 –Accuracy 0-5 Assessment –Chosen Items to assess 0-5 –Accuracy 0-5 Communication –Communication techniques 0-5 –Professionalism 0-5 Critical Thinking –Decisions and Interventions Chosen 0-5