1. Belal M. Hijji, RN. PhD, Arwa Owies, RN. PhD Faculty of Nursing, Philadelphia University, Jordan Presented in The second International Conference for Allied Health Care Professionals, Physicians, Physiotherapists, and Technicians, April 7- 8 2010, Alexandria, Egypt An Examination of the Routine Blood Transfusion Knowledge Questionnaire Through Item Analysis

2. Background Aim and objectives Methods Results Summary of results and discussion Conclusion Acknowledgment References Contents

3. 8.04.10The second International Conference for Allied Health Care Professionals 3 Background

4. 8.04.10The second International Conference for Allied Health Care Professionals 4 The Routine Blood Transfusion Knowledge Questionnaire (RBTKQ) was developed (Hijji, 2007) as a measure to investigate nurses’ level of knowledge and practice of blood transfusion The RBTKQ includes has true-false items (three), multiple-choice (MC) (20), and multiple-response (MR) (10) For the purpose of this analysis, five sections of the RBTKQ that are relevant for inclusion: (A) is about issues related to patient preparation before blood bag collection, (B) is concerned with blood bag collection section, (C) is related to pre-transfusion initiation nursing activities, (D) addresses post- transfusion initiation activities, and (E) is related to complications of blood transfusion

5. 8.04.10The second International Conference for Allied Health Care Professionals 5 An examination of empirical item response data maybe appropriate in detecting item flaws (Crocker and Algina, 1986), and provides useful information about test-item quality (Oermann & Gaberson, 1998). Since its inception, the RBTKQ has not undergone any form of item analysis to evaluate its psychometric properties

6. 8.04.10The second International Conference for Allied Health Care Professionals 6 AIM

7. 8.04.10The second International Conference for Allied Health Care Professionals 7 Principal Aim The main aim of this study was to determine the overall quality of the items and the test Objectives: A. Identify the items’ difficulty “p” and discrimination “D” levels B. Investigating the performance of distractors C. Find out whether the items discriminate between nurses whose scores place them in the top 30% of all nurses and those whose scores place them in the lower 30% of all nurses. This figure provides a stable index of differences between high and low ability groups (Crocker & Algina, 1986)

8. 8.04.10The second International Conference for Allied Health Care Professionals 8 Methods

9. 8.04.10The second International Conference for Allied Health Care Professionals 9 Data Collection Responses to the item were taken from the RBTKQ that was filled out by a random sample of 305 nurses (95.3%). Analysis was performed for p, D, and for the distractors. Because the p value is unavailable with MR items ( http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php), each one of these was broken down to a number of sub-items equal to the number of its options. Based on a nurse’s selection of an option, a label of “yes” or “no”, or “true” or “false” was assigned. The outcome of this procedure culminated in a total of 89 items for the whole questionnaire. http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php

10. 8.04.10The second International Conference for Allied Health Care Professionals 10 Data Analysis scoring For most items, one point was awarded for the correct answer and 0 for the incorrect one. The maximum possible score was 57 (100%) points The number of nurses’ subpopulations was 91 for each group. The upper group’s scores ranged from 32 (56%) to 40 (70%); lower group from 8 (14%) to 27 (48%). The mean knowledge score for the sample is 29.6 (SD = 4.14). A passing score of 34 (60%) (one standard deviation above the mean) was set up. Scores were arranged in descending order from the highest to the lowest

11. 8.04.10The second International Conference for Allied Health Care Professionals 11 For each item, data analysis details: A.the p value (whole sample). The p value is set at 0.5 to 0.85 (Lin et al., 1999). B.the p value of each option selected by both groups C.the difference in proportions “D” between nurses’ subgroups selecting each option. A D value of 0.20 or above is appropriate (Crocker and Algina, 1986; Brown, 1983) for the correct option. D.the number of omits

12. 8.04.10The second International Conference for Allied Health Care Professionals 12 Results

13. 8.04.10The second International Conference for Allied Health Care Professionals 13 Due to time restrictions, there will be partial presentation of the results

14. 8.04.10The second International Conference for Allied Health Care Professionals 14 Table 1. Computations of item analysis indices for sections A and C Lower 30%Upper 30%Percentages Section A Options p OmitFTKeyItem.471-155-2835-62F 1.60-.31.40-.69%.29-.29 D (h – l).303-235-2453-65F 2.38-.26.61-.71%.12-.13d.331-541-2249-64F 3.45-.24.55-.76%.21-.21d Section C.890-04-1987-72T1.04-.21.96-.79% -.17.17d.810-078-6613-25F2.86-.73.14-.27%.13-.13d [1] [1] Difficulty index for the top and lower 30% of nurses [2] [2] This is the difference between the proportions of nurses’ subpopulation. The h stands for high scorers; l for low scorers

15. 8.04.10The second International Conference for Allied Health Care Professionals 15 Table 2. Computations of item analysis indices for section D D Lower 30%Upper 30%Percentages: Options pOmitFTKeyItem.330-152-7139-19T1.57-.79.43-.21% -.22.22d.830-113-1578-75T2.14-.18.86-.82% -0.04.04d.680-168-5223-38F3.75-.57.25-.43%.18-.18d.470-150-3641-54F4.55-40.45-.60%.15-.15d.710-114-44 77-46 T5.15-.49.85-.51 % -.34.34 d

16. 8.04.10The second International Conference for Allied Health Care Professionals 16 Table 4. Computations of item analysis for sections A, B, and C Lower 30% Upper 30% Percentages Section A Options p OmitEDCBAKeyItem.120-1--22-47-135-757-66D 4.24-.05.08-.14.05-.08.63-.73%.19-.06-.03-.10d Section B.511-5-2-331-471-1056-26A1.03-.09.34-.52.01-.11.62-.29% -.06-.18-.10.33d.460-111-1629-252-81-648-35A2.12-.18.32-27.02-.09.01-.08.53-.38% -.06.05-.07.15d.092-16-110-3711-76-636-40C3.07-.01.11-.41.12-.08.09-.08.40-.44%.06-.30.040.01-.04d [1] [1] Omits were considered incorrect responses (Oermann & Gaberson, 1998). Here omits were applied to option D.

17. 8.04.10The second International Conference for Allied Health Care Professionals 17 Table 4. (Continued…….) Section C Options p OmitEDCBAKeyItem.300-05-64-339-158-1635-51 C 1.05-.07.04-.03.43-.16.09-.18.38-.56% -.02.01.27-.09-.18d.380-1 0-3 0-921-3219-2551-22A2 0-.03 0-.11.23-.35.21-.27.56-.24% -.03 -.11-.12-.06.32d.081-1 0-0 13-72-819-3056-45D3.14-.08.03-.10.21-.33.62-.49%.06-.07-.12.13d.901-0 88-70 0-31-50-71-6E4.97-.77 0-.03.01-.050-.08.01-.07%.20 -.03-.04-.07-.06d.2713-10 - 7-1325-1416-2230-33C5.22-.25.27-.15.18-.24.33-.36% -.03.12-.06-.03d.701-0 - 0-481-503-156-22C6 0-.04.89-.55.03-.16.07-.24% -.04.34-.13-.17d

18. 8.04.10The second International Conference for Allied Health Care Professionals 18 Table 5. Computations of item analysis indices for sections D Lower 30% Upper 30% Percentages Options pOmitEDCBA KeyItem.323-1 - 4-128-1531-2137-37B1.08-.14.09-.16.34-.23.41-.41% -.06-.07.110d.051-5 9-2 1-32-54-1274-64E2.10-.02.02-.09.02-.05.04-.13.81-.70%.08 -.07-.03-.09.11d.141-2 - 4-68-1020-1748-42C3.05-.09.10-.13.25-.22.59-.55% -.04-.03.03.04d.451-2 - 8-861-3015-176-34C4.10-.11.67-.33.16-.19.07-.37% -0.01.34-.03-.30d.450-4 - 13-1228-2341-379-15B5.14-.13.31-.25.45-.41.10-.21%.01.06.04-.11d [1] [1] Item not applicable to 22 nurses [2] [2] Item not applicable to 76 nurses

19. 8.04.10The second International Conference for Allied Health Care Professionals 19 Table 5. Computations of item analysis indices for sections E Lower 30%Upper 30%PercentagesItem Options pOmit E DCBAKey.500-0 - 11-1559-345-2916-13C1.12-.16.65-.37.05-.32.18-.14% -.04.28-.27.04d.450-2 14-17 7-2151-3111-158-5C2.15-.19.08-.23.56-.34.12-.16.09-.08% -.04 -.15.22-.04.01d.090-3 0-1 4-69-53-1375-63C3 0-.01.04-.10.10-.05.03-.14.82-.69% -.01 -.06.05-.11.13d.293-2 - 44-1527-4115-212-12D4.48-.16.30-.45.16-.23.05-.15%.32-.15-.07-.10d.650-3 - 1-66-3179-395-12B5.01-.10.07-.34.87-.43.05-.13% -.09-.27.44-.08d

20. 8.04.10The second International Conference for Allied Health Care Professionals 20 Summary of Results and Discussion Item difficulty –Eight items (27%) had p value of.50 to.85 (neither too difficult nor too easy) –Two items (6%) had p value of >.85 (easy) –20 items (67%) had p value of <.50 (difficult). This could be, most probably, because nurses did not master the content All distractors were selected by nurses and only a few of them, sometimes, omitted responding to some items. Sixteen items (53%) had a D value of ≥.20. This indicates that the items were especially easier for the top group and more difficult to the lower group. –The last two results are good indicators of the quality of item (Osterland, 1998, Oermann & Gaberson, 1998). Functional distractors decrease the chances that low ability nurses select the correct response by guessing which would introduce measurement error (Crocker & Algina, 1986)

21. 8.04.10The second International Conference for Allied Health Care Professionals 21 Fourteen items (47%) had a D value of <.20. –The test was not designed to maximise variance among groups. In essence, D relies on a fundamental assumption, which is that examinees who exhibit mastery on the subject matter are presumed to be more likely to respond correctly to an item than low ability examinees (Osterland, 1998). In this study in fact, none of the nurses per se mastered the content of interest. The differences between both groups for the distractor were, mostly, negative values. This indicates that fewer upper group nurses selected the distractor than did the lower group nurses. –This is another positive feature of items (Osterland, 1998) However, the differences for some distractors in eight items were scant (≥ -.04). For two items, the differences were positive indicating that higher ability nurses selected distractors more often than low ability nurses. –In this case, Osterland (1998) recommends reviewing the options for possible improvement.

22. 8.04.10The second International Conference for Allied Health Care Professionals 22 For a number of items (n = 13, 43%), nurses responses were widely scattered among the responses’ alternatives, and/ or the correct responses attracted less attention than the distractor(s). –This result may indicate that nurses could not understand the items or that the items covered unfamiliar content. Therefore, these items may need to be reviewed (Crocker & Algina, 1986).

23. 8.04.10The second International Conference for Allied Health Care Professionals 23 Conclusion This study has highlighted the need for: –Jordanian nurses to master the content related knowledge and practice of blood transfusion. –Examining the questionnaire items for possible improvement However, no educator should retain, revise, or eliminate an item on the basis on item statistics alone (Oermann & Gaberson, 1998).

24. 8.04.10The second International Conference for Allied Health Care Professionals 24 Acknowledgements

25. 8.04.10The second International Conference for Allied Health Care Professionals 25 We are immensely grateful to the Ministry of Health, all nursing staff, and hospitals’ management. We express our gratitude to Philadelphia University Deanship of Scientific Research and Postgraduate Studies for funding the study.

26. 8.04.10The second International Conference for Allied Health Care Professionals 26 References

27. 8.04.10The second International Conference for Allied Health Care Professionals 27 Brown, F. (1983). Principles of educational and psychological testing (3rd ed.). New York: Holt, Rinehart and Winston. Crocker, L. and Algina, J. (1986). Introduction to classical & modern test theory. Belmont CA: Wadsworth. Hijji, B. (2007). Knowledge and practice of blood transfusion: A survey of nurses in Abu Dhabi, United Arab Emirates. Unpublished PhD thesis, the University of Ulster. Lin, L.; Tseng, H. and Wu, S. (1999). Item analysis of the registered nurse licensure exam taken by nurse candidates from vocational nursing high schools in Taiwan. Proc. Natl. Sci. counc. ROC (D). 9(1): 24-31. Oermann, M. and Gaberson, K. (1998). Evaluation and testing in nursing education. New York: Springer. Osterland, S. (1998). Constructing Test Items: Multiple-Choice, Constructed- Response, Perfromance, and Other Formats. 2 nd edition. London: Kluwer.

28. 8.04.10The second International Conference for Allied Health Care Professionals 28 Thank You Principal Investigator bhijji@philadelphia.edu.jo bhijji1@gmail.com