Affect of Viewing Tutorials on Percentage Change: Understanding How To Evaluate Information: A Three-Semester Study of Early WSU Undergraduates Steve Borrelli Instructional Design Librarian Corey Johnson Head, Library Instruction Washington State University Library Instruction Team Introduction: This poster outlines a multiple term collaborative effort in the area of information literacy, targeting student evaluation skill development. Librarians taught hybrid (mostly online with some in-person interaction) lessons to groups of lower undergraduates enrolled in Pathways to academic Success Seminar (PASS). A pre-test/post-test assessment package revealed that students’ strongest area of proficiency was in understanding the differences between popular and scholarly sources, with lowest performance concerning understanding the components of a scholarly article. It was also concluded that the academic term with the strongest collaborative ties between librarians and seminar instructors/leaders resulted in the greatest improvement in student learning across all measures. Approach: (1) Between the pre-test and post-test students answered 20 questions focused on evaluating information. The twenty questions were mapped to the three areas of: -The differences between popular and scholarly articles -Components of a scholarly article -Methods of evaluating scholarly content online (2) Tutorial access was tallied for each student (3) Calculations for percent change were conducted to determine increases in student learning between pre- and post-tests Discussion: Participation: -Strong collaborative ties result in student learning -The fall 2010 semester: -Strongest student participation -Largest increases in student learning across all three areas -Lowest percentage of students not viewing any tutorials & lowest percentage of students whose scores decreased or remained the same -Student engagement with the curricular content impacts student learning -Students who viewed at least one tutorial showed percentage increases larger than those who did not view any Student learning: -All 3 semesters saw increases in student learning in each area assessed -Pre- & post-test scores indicate that students do understand the difference between popular and scholarly content -Percent increases between the pre- & post-tests show that students struggle understanding components of a scholarly article -Of the assessed areas, students showed the greatest gains in the area of evaluating scholarly content online Collaborative History: Spring 2010: -Librarians went to each PASS section classroom, introduced the pre-test monitoring students through completion, & briefly explained the content of questions where students performed the worst -Librarians provided PASS a summary document about student performance and participation Fall 2010: -Ability to monitor which students accessed each tutorial, added to ILE at the request of PASS instructors -PASS extended post-test due dates to get more participation Spring 2011: -PASS Instructors administered the Pre- & Post-tests -Librarians provided PASS with periodic information about student tutorial accesses Results : Spring 2010: -264 students completed both tests, of those: -63% increased in score -16% did not change -21% of decreased in score -Average number of tutorials viewed per student completing both tests = 1.95 -Average percent increase was 29.98% -Average number of tutorials viewed for students taking at least one tutorial completing both tests = 3.35 -Average percent increase for students viewing at least one tutorial 74.75% Fall 2011: -90 students took both tests, of those: -80% increased in score -14% did not change -06% decreased in score -Average number of tutorials viewed per student completing both tests = 2.88 -Average percent increase was 62.46% -Average number of tutorials viewed for students taking at least one tutorial completing both tests = 3.45 -Average percent increase for students viewing at least one tutorial 91.45% Spring 2011: -69 students completed both tests, of those: -67% increased in score -12% did not change -22% decreased in score -Average number of tutorials viewed per student completing both tests = 0.64 -Average percent increase was 31.72% -Average number of tutorials viewed for students taking at least one tutorial completing both tests = 2.04 -Average percent increase for students viewing at least one tutorial 49.31% Literature Review: Information literacy instruction has recently gained an elevated priority in academia (Klusek & Bornstein), while studies have shown development of these skills to be lacking (Foster 2006, Cerny-Koenig, et. al. 2007). Information evaluation is a key information literacy skill. Most of the literature related to teaching college-level information evaluation skills focuses on Web site evaluation (Head & Isenberg) and studies utilizing student paper citation analysis (Davis & Cohen, Jamieson & Howard). Although this study addresses Web site evaluation, it also uniquely assesses student ability to understand differences between popular and scholarly articles and components of a scholarly article. Affect of Viewing Tutorials on Percentage Change: Spring 2010 Fall 2010 Spring 2011 Average number of tutorials taken per student 1.95 2.88 0.64 Average number of tutorials taken for students taking at least 1 tutorial 3.35 3.45 2.04 Average percent increase for students taking at least one tutorial 74.75% 91.45% 49.31% Conclusions: -The collaboration resulted in student learning in each semester -Further evidence that focused exposure to the concepts and ideas of evaluating information results in student learning -The more coordinated relationship in fall 2010 resulted in the largest increases in student learning -Student learning is impacted by motivation, vesting the student with an adequate reward for the effort may result in increased engagement with the curricular content -Students in research institutions are often tasked with integrating scholarly research into their work, students would benefit from being introduced to carefully selected scholarly material early in their education -If students demonstrate an understanding of the differences between popular and scholarly information yet continue to rely on quickly available web resources, then more research may be needed in the area of appropriately motivating students when collaborating with libraries WSU Seven Learning Goals: Critical & Creative Thinking Quantitative Reasoning Scientific Literacy Information Literacy Communication Diversity Depth, Breadth, and Integration of Learning Bibliography: -Cerny-Koenig, T., Perillo, C. A., Pearson-Mims, C. H., Williams, K. M., Brown G., Morozov A., Pan, W. L., & Hendrix, W. (2007). Piloting a program-level learning assessment plan in plant and soil science. North American Colleges & Teachers of Agriculture Journal. 51(3), 26-33. -Davis, P.M. & Cohen, S.A. (2001). The effect of the Web on undergraduate citation behavior, 1996 – 1999. Journal of the American Society for Information Science and Technology. 52(4), 309-314.. -Foster, A. (2006). Students fall short on ‘information literacy,’ educational testing services study finds. Chronicle of Higher Education, 53(10), A36. -Head, A. J., & Eisenberg, M. B. (2010). Truth be told: How college students evaluate and use information in the digital age. Project Information Literacy Progress Report, University of Washington's Information School. -Jamieson S., & Howard, R. M. (2011). The citation project findings: Sources cited in Paper. -Klusek L., & Bornstein J. (2006). Information literacy skills for business careers: Matching skills to the workplace. Journal of Business & Finance Librarianship, 11(4), 3-21. Information Literacy at WSU: Graduates will effectively identify, locate, evaluate, use responsibly and share information for the problem at hand. For instance, graduates can demonstrate information literacy by their ability to: Determine the extent and type of information needed Implement well-designed search strategies Access information to accomplish a specific purpose Assess credibility and applicability of information sources Use information to accomplish a specific purpose Access and use information ethically and legally