Observations as a Clinical Evaluation Method in Nursing Education Dr. Belal Hijji, RN, PhD 22/05/2011

Learning Outcomes At the end of this presentation, participants will be able to: Discuss the value of observation in clinical evaluation. Identify some of the threats to the reliability of observations. Identify the advantages of using anecdotal notes and checklists in assessment of performance. Describe how to use anecdotal notes and checklists as methods of recording observations about student’s performance. 2

Observations “For as long as people have been interested in studying the social and natural world around them, observation has served as the bedrock source of human knowledge”. (Adler and Adler, 1994, p. 377) Observing students is a main strategy for evaluating performance in clinical practice, simulation, and learning laboratories. (Oermann and Gaberson, 2009) 3

Threats to Validity and Reliability of Observations Observations of students may be influenced by the teacher’s values, attitudes, and biases. Different teachers may focus attention on different aspects of performance. For example, during an IV medication administration, a teacher may focus mainly on the technique used for its administration, while another may focus on asking questions about the purpose of the medication, and observing how the student interacts with the patient. Incorrect judgments could be made about student’s performance, such as inferring that a student is inattentive during conference when in fact s(h)e is thinking about the comments made by others. 4

All observed behaviors should be shared with the students. The level of student’s performance may vary between observations. This also applies to teachers and nurses. All observed behaviors should be shared with the students. 5

Recording Observations Recording observations of students in clinical settings, simulation, and learning laboratories could be carried out through anecdotal notes, checklists, and rating scales. Anecdotal notes and checklists will be the focus of this presentation. 6

Anecdotal Notes Teachers may not remember the observations made of each student for each clinical activity. Anecdotal notes are helpful in remembering what they observed and the context in which the performance occurred. The elements of effective anecdotal notes are presented below. Figure 1. Key elements of effective anecdotal notes 7

Figure 2. Poorly written anecdotal note Anecdotal notes are narrative descriptions of observations made about student’s performance. Figure 2. Poorly written anecdotal note 8

Figure 3. Well-written anecdotal note 9

Anecdotal notes should be recorded as close to the time of the observation as possible. Anecdotal notes should be shared with students as frequently as possible; otherwise they are not effective for feedback. 10

Checklists A checklist is a list of specific behaviors to be observed with a place for marking whether or not they were applied during the performance (Slide 12). Nurse educators use or develop checklists that often list the steps a student should follow when performing a nursing procedure or demonstrating a skill. Checklists facilitate the teacher’s observation of procedures and students’ behaviors and learning new skills, and checklists also provide a way for learners to assess their own performance. Checklists enable learners to review and evaluate their performance prior to assessment by the teacher.

A Checklist for IV Injection of Medication Step Activity Applied Not applied 1 Checks physician’s order 2 Checks medication administration record 3 Adheres to rights of medication admin. 4 Assembles appropriate equipment 5 Checks compatibility with existing IV if present 6 Explains procedure to patient 7 Positions patient appropriately 8 Checks patency of IV line 9 Administers medication at proper rate 10 Monitors patient response 11 Flushes tubing as necessary 12 Documents IV medication correctly

Checklists (Continued…) It is favorable that a checklist include errors in performance that are commonly made. For example, Hijji et al.’s (2010) observational study on nurses’ observed practice of blood transfusion (n = 49) included the following 3 errors in practice: Practice Performed N % Did not perform N % Covered blood with blanket/ linen 7 14 42 86 Performed remote checks at nurses’ station 30 61 19 39 Initiated the transfusion more than 30 minutes after blood collection 21 43 28 57

Regardless of the method being used, educators should record observable and objective NOT subjective data, and should have the knowledge of how to develop valid and reliable methods to assess students’ performance. For example, in a previous experience of the Presentor in a Middle Eastern university, nurse educators included the following multivariable item in their assessment of students’ performance on “establishing a sterile field”: Professional development, communication skills, attendance, and punctuality. on a scale of 1 2 3 4 5 6 7

References Adler, P. and Adler, P. (1994). Observational techniques. In N. Denzin and Y. Lincoln (eds.) Handbook of Qualitative Research. London: Sage. pp. 377-392. Hijji, B., Parahoo, K., Hossain, M., Barr, O. and Murray, S. (2010). Nurses’ practice of blood transfusion in the United Arab Emirates: an observational study. Journal of Clinical Nursing. 19, 3347–3357. Oermann, M. & Gaberson, K. (2008). Evaluation and Testing in Nursing Education. 3rd edition. New York: Springer. Nitko, A. J., & Brookhart, S. M. (2007). Educational assessment of students (5th ed.). Upper Saddle River, NJ: Pearson Education.

Thank You Very Much Any Questions?