1. Making a difference? Measuring the impact of an information literacy programme Ann Craig a.craig@worc.ac.uk

2. Structure of presentation Context and rationale Literature Methodology: –Pre- and post-tests and interviews Results: –Increased skills and confidence? Outcomes Recommendations

3. Context University of Worcester Student profile: –Approx 7500 students –60% mature students –Non-traditional entrants

4. Context Information literacy programme –Embedded across three years of pre-registration Nursing course –3 key sessions in first semester 2 whole group sessions & 1 set of small group sessions Small team of subject librarians: 36 hours of contact time with students during first semester –Two intakes each year (September and February) Evaluation methods were mainly subjective

5. Research questions How effective is the programme in developing students’ skills? Does the programme increase students’ confidence? Are the factors of skill and confidence linked? How do demographic characteristics relate to skill and confidence?

6. Aims and objectives Aim: to investigate whether the information literacy programme for pre-registration nursing students is effective in increasing skills and confidence –To test students’ skill levels before and after two key sessions –To explore factors affecting confidence –To identify any relationships between skills, confidence and demographic characteristics Aim: to inform the development of information literacy at the University of Worcester

7. Literature: key themes Assessment of information literacy –Range of methods employed Skills: tests –Fox et al (1996), Shorten et al (2001), Brettle (2003) Confidence –Kurbanoglu (2003) and Ren (2000) Demographic factors

8. Literature: key themes Evaluations of programmes for pre-registration nursing students: –Range of methods used Little evidence that programmes for nursing students are effective Lack of published instruments

9. Methodology Skills of cohort tested using pre- and post- intervention questionnaire (quiz) : –To measure change –14 multiple choice questions: only one correct answer –Same questions used for both tests, in same order –Likert scale question(s) about confidence –Information on age and gender

10. Methodology Confidence was explored using a semi-structured interview Theoretical sampling for divergent cases Questions covered: –Educational level, previous experience of skills programmes –Confidence before, during and after programme –Increase or decrease in confidence during programme –Factors influencing confidence at all stages

11. Research Study 70 students completed pre-test Change in attendance policy: only 32 students completed post-test 29 students completed both –Mean age: 32 –83% female & 17% male Impact on test results Impact on selection of students for interview: 5 interviews completed

12. Results: Skills Change in group is statistically significant: p<0.05 (Wilcoxon Signed Ranks Test) Mean number correct (out of 14) –Pre-test9.10Whole group 8.44 –Post-test10.59 Mean change+1.48 21 students scored more in post-test 5 students showed no change 3 students scored less in post-test

13. Results: Skills

14. Most correct answers: use of library catalogue, refining searches and use of search logic –Little significant positive change can be demonstrated when students scored highly at the pre-test Fewest correct answers: use of specific techniques e.g. identifying keywords Selection of “Internet” as answer for information sources questions

15. Results: Confidence 5 interviewees: all female and 31 or over Representative of range of levels of confidence Confidence often dipped during programme All interviewees were “Fairly Confident” or “Very Confident” by end of programme Comparison with self-assessment in tests

16. Results: Confidence Factors affecting confidence: + Programme itself: particularly small group sessions + Previous successful use of IT: “Mastery experiences” + Experience, and confidence in, using libraries - Lack of experience with IT - Problems with passwords!

17. Results: Confidence

18. “I think I was a bit naïve actually...I thought you’d be attending your lectures…Nowhere was I thinking it’s access wwwdotslash. It completely put me on edge.” “I was quite proud of myself for what I’ve managed to achieve.”

19. Relationship between skills and confidence Evidence of a link at the pre-test Some over assessment of confidence by the post-test: some confidence ratings were not justified Interview sample: assessed confidence in line with skill level

20. Demographic characteristics Sample for pre- and post-tests was too small to draw valid conclusions Whole group pre-test indicated that students aged 18-30 were marginally more confident than other students No difference in scores between age groups No evidence of differences due to gender: only 5 male students

21. Outcomes Tests provided evidence that sessions should be made mandatory Tests highlighted areas of difficulty for whole group: inform planning More practical exercises in whole group sessions and brief tours in the induction Highlighted need to stress relevance of sessions – working with colleagues Tests have been reused

22. Recommendations Longitudinal study though whole course Conduct study with comparison group Extend use of tests: other disciplines New Nursing curriculum: challenges –Less face to face contact –Creation of interactive materials via WebCT –Student support

23. Summary: key results Programme is effective in increasing students’ skills and confidence Confidence levels rose by the end of the programme Some evidence of a link between skills and confidence No firm evidence of the impact of demographic characteristics on skills and confidence Future challenge: providing same learning opportunities online

24. References Brettle, A. (2003). “Information skills training: a systematic review of the literature.” Health Information and Libraries Journal. 20 (Suppl 1.), 3-9. Fox, L., Richter, J.M. and White, N.E. (1996). “A multidimensional evaluation of a nursing information-literacy program.” Bulletin of the Medical Library Association. 84(2), 182-190. Kurbanoglu, S. (2003). “Self-efficacy: a concept closely linked to information literacy and lifelong learning.” Journal of Documentation. 59(6), 635-646. Ren, W.H. (2000) “Library instruction and college student self- efficacy in electronic information searching.” Journal of Academic Librarianship. 26(5), 323-328. Shorten, A., Wallace, M.C. and Crookes, P.A. (2001). “Developing information literacy: a key to evidence-based nursing.” International Nursing Review. 48, 86-92.