1. LIBQUAL+® AND THE EVOLUTION OF LIBRARY AS PLACE AT RADFORD UNIVERSITY, 2002-2008 Eric Ackermann Radford University Library Assessment Conference University of Washington August 3-7, 2008

2. Introduction Me, the 15% Assessment Librarian Radford University McConnell Library

3. Background Changes to Library as Place continuous since 2002 Examples Coffee shop Designated quiet areas Group study areas No cell phone zones New furniture Improved signage

4. Goal of this study Are these physical changes reflected in our LibQUAL+® survey results?

5. Study design Data types: Score (rankings) & comments Data source: 2002, 2005, 2006, 2008 LibQUAL+® surveys Respondent groups: Faculty Graduate students Undergraduates Level of analysis: Library as Place dimension

6. Methodology Meta-analysis A data reduction method used to combine results across studies using Effect size meta-analysis Statistical procedure that converts incomparable metrics into a common effect size metric.

7. Data analysis Organize the data & respondent types into comparison groups Faculty: 2006 v. 2002 Grad students: 2005 v. 2002, 2008 v. 2005 Undergraduates: 2005 v. 2002, 2008 v. 2005 All groups: 2005/6 v. 2002, 2008 v. 2005/6

8. Data analysis (cont) Determine effect size for each comparison group using mean adequacy gap (score data) odds ratio (positive: negative comments) Determine final weighted average effect size d for each respondent group for 2002-2008 Results reported as d,.95 CI, and BESD

9. How did we measure success? (aka, Practical significance)

10. Practical significance Success LevelSignificance Action criteria Action dBESD Complete Users satisfied 0.31 or more 15.1% or more Celebrate! Partial Users somewhat satisfied 0.3 to -0.3 15% to -15% Look for improvements; implement changes Un- successful Users unsatisfied -0.31 or less -15% or less High Priority review/change

11. What did we expect to find?

12. In the literature… Nothing found about specifically using LibQUAL+® to measure satisfaction with changes in Library as Place All reports found about measuring change due to drastic overhaul of building or new construction.

13. Conclusion… If dramatic changes to the Library as Place yield dramatic, immediate increases in user satisfaction, then incremental changes will yield modest gains in user satisfaction over time.

14. What did we actually find?

15. Response rate & representativeness Response rate (average of 2002-2008) Faculty: 15.7% Grad students: 22.4% Undergraduates: 26.4% Representativeness (average of 2002-2008) Faculty: 8.2% Grad students: 7.4% Undergraduates: -13.1%

17. So what? (Practical significance again)

18. Practical significance Partial success (Users somewhat satisfied) Action Criteria: d + 0.30 (or BESD + 15%) Faculty (0.01 or 0.6%) Grad students (0.19 or 9.2%) Undergrads (0.08 or 4.1%) Action Looked for improvements (e.g., examined Suggestion coded comments, conducted focus groups) Implemented changes (see next slide…)

19. Review/change Improvements since 2008 LibQUAL+ survey Created Front Desk from Media Services and Circ desks Reorganized quiet study floor Reduced noise level on quiet study floor More improvements planned New furniture for the lobby Walls to be painted Reference area reorganization

20. Conclusion Effective Method is statistically defensible Told us what we did (and did not) want to hear Practical Metrics have practical meaning at the local level Provide criteria for actionable results Sustainable Commitment to assessment by library administration Funded by university

21. Questions?

22. Contact information Eric Ackermann Reference/Instruction and Assessment Librarian egackerma@radford.edu 540-831-5488 Box 6881, McConnell Library, Radford University, Radford, VA 24142