1. Informing Reference Services @ Columbia University Libraries Using Today’s Numbers to Plan Tomorrow’s Services RUSA Program, ALA Annual Conference 4:00PM, June 26,2011 Jennifer Rutner, Assessment & Planning Librarian Columbia University Libraries

2. Reference Assessment @ Columbia Digital Centers User Needs Assessment Service Evaluation Virtual Reference Assessment

3. Assessment Program Mission Statement Serve library users and staff through the gathering, analysis, and application of high- quality, actionable information to guide library decision making.

4. Project Teams and Decision Making Appoint project team Conduct assessment Make recommendations Implementation

5. Defining Information Needs Identify existing data Brainstorm questions Information needs “IWTK” Prioritize information needs Determine audience Assign methodology Develop protocol

6. Library vision User needs

7. Assessing IM Reference

8. IM at Columbia Libraries CUL launches Chat Reference 2001 Evaluation of Virtual Reference transactions 2006 Switch to Meebo Departmental library accounts 2007 User Assessment 2009 Switch to LibH3lp Expand staffing and hours, consolidate services 2010

9. Motivation Understand how IM services are being used. Evaluate quality of service provided via IM reference services. Identify user-needs for reference support. Clients Reference Coordinating Committee IM Reference Coordinator

10. Methodology Survey Service evaluation survey sent to IM users Focus groups Students who had never used IM reference Libraries staff Data Analysis Compilation of IM statistics from 2007- 2009

11. Findings: User Survey 46% were graduate students 41% were between 23-29 years old 61% were 1 st time library IMers 79% discovered IM through the website 51% were at home when they IM’d

12. Findings: User Survey 74% were very satisfied with the information they received when using the IM service 69% strongly agreed that the resources suggested by the librarian were useful 85% strongly agreed that using IM to contact a librarian was very easy 80% agreed that the hours were satisfactory

13. Findings: Focus Groups Hours: “It’s not 24 hours? That sucks.” Communication: “It is a little bit awkward, but it’s good that librarians are taking this to the instant messaging level. It’s just better service.” Marketing: “Very visible… if you’re trying to eventually expand the hours and make it more visible for students…” Staffing: “If they could answer the question, whoever can answer the question.” Convenience: “I live off campus so, my Columbia experience is minimal. So, this IM thing would probably be helpful for someone like me, who doesn’t chill in the library.”

14. Findings: Transaction Analysis 60% were policy/procedural, research or holdings questions 8% were e-resource problems 90% come through the widget 57% required no follow-up 85% are <10 minutes 1702 transactions logged from 2007-2008.

15. Recommendations An ideal IM service for our users would be a 24/7 service; staffed by knowledgeable, friendly people; easy to access from the Libraries’ web pages that they use most; where they can get quick efficient service.

16. Impact: Service Library-wide participation MLS interns 15 additional staff Staffing 36 additional hours/week Hours Original: Live Assistance Interim: Meebo Today: Library H3lp Platform

17. Impact: Usage Year# of IM Transaction 2007468 2008383 2009549 20102293 2011 (through May)1722

18. Digital Centers: User Needs Assessment

19. Locations Digital Social Science Center @ Lehman Social Science Library Digital Humanities Center @ Butler History & Humanities Library Digital Science Center @ Science & Engineering Library The Burke Library @ Union Theological Seminary Avery Architectural & Fine Arts Library

20. Motivation How is the changing information environment impacting library use, and student needs for technology and research support? Clients Departmental libraries Libraries IT Office Library Leadership

21. Methodology Social Sciences 2007 Online questionnaire Student focus groups (2) Faculty interviews Observation studies Humanities 2009 Online questionnaire In-library paper questionnaire Focus groups, TBD Sciences 2010 Online questionnaire In-library paper questionnaire In-library flip charts

22. Results Social Sciences Survey n = 125 Humanities Survey n = 940 Sciences Survey n = 611

23. Impact: Digital Social Science Center From on-call to desk hours Mode Increased visibility Centralized service point Location Promote consultation services “Info Expo” event Outreach Group study Presentation practice room Study space Presentation practice room Quantitative support, GIS, bibliographic software Technology

24. Ongoing! Impact: Digital Humanities Center Dedicated 2 reference staff to the DHC Staffing Transitioning reference support from the reading room to the DHC Location SCANNERS! Technology Extended hours in the DHC Late night vs. overnight Hours

25. Impact: Digital Science Center Increased reference visibility Location Emerging Technology Librarian hired Staffing Group study rooms and booths Study space Quantitative software Visualization software Technology More workshops Instruction

26. Digital Centers: Evaluation

27. DSSC Evaluation Motivation Understand awareness of and satisfaction with new services at the DSSC. Methodology Online survey In-library paper survey Response 416 respondents – 47% Graduate Students – 80% from the social sciences

28. Findings: Technology 66% “never heard of” data services, Bloomberg stations 47% “heard of/never used” GIS Overall satisfaction with technology services

29. Findings: Research Support 34% “never heard of” the reference desk 42% “never heard of” emailing a librarian 50% “never heard of” of research consultations 50% “never heard of” library workshops 66% “never heard of” IMing a librarian

30. Findings: Space 75% use the library for individual study 68% use the library for group study 67% are satisfied with individual study 67% are satisfied with group study 61% “never heard of” practice presentation room 73% “never heard of” conference calling room

31. Impact Continue renovations Improve maintenance Facilities Continue to expand group study Study space Promote IM Promote GIS/Data services Outreach Offer more workshops New orientation approach Instruction

32. Thank you. jenrutner@columbia.edu Special thanks to: Amanda Bielskas, Rob Cartolano, Kathleen Dreyer, Mary Giunta, Jean LaPonce, Bob Scott, Jane Winland