Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective Assessment Phase I Update ARL VISITING PROGRAM OFFICERS Steve Hiller Library Assessment Coordinator, University of Washington Jim Self Director, Management Information Services, University of Virginia & Martha Kyrillidou ARL, Director of Statistics and Measurement Program

ARL New Measures Key Areas User Satisfaction/Service Quality –LibQUAL+™ – Measuring Library Service Quality Lyceum Market Penetration Ease and Breadth of Access Library Impact on Teaching and Learning –Project SAILS, Learning Outcomes Library Impact on Research Cost Effectiveness Library Facilities/Space E-Metrics –MINES for libraries –ARL supplementary statistics

Barriers to Management Information and Using Data Effectively in Libraries Organizational culture/leadership support Time/Staff/Resources Data issues – Too much, compatibility, validity Establishing priorities Knowing what to measure and methods to use Inexperience, perceived lack of skills and expertise Understanding, presenting and knowing what to do with the results Hiller, S. and Self, J. (2004). From Measurement to Management: Using Data Wisely for Planning and Decision-Making. Library Trends.

Making Library Assessment Work: Practical Approaches for Developing and Sustaining Effective Assessment 2 year ARL project to assist libraries with moving assessment efforts forward Led by Steve Hiller (UW) and Jim Self (UVa) Recognition that libraries understand value of assessment but have trouble sustaining efforts and using results Site visits conducted to evaluate assessment and develop practical approaches to effective local practices Identify common barriers and facilitators to assessment 7 libraries participating in Phase I Spring 2005; during academic year; final report in 2006

Phase I Participants University of ArizonaFebruary 2005 Arizona State UniversityFebruary 2005 University of ConnecticutMarch 2005 University of Illinois at UCMay 2005 New York UniversityMarch 2005 University of Notre DameJune 2005 University of OregonMay 2005

Phase I Process Each library chooses a contact person Survey sent to contact person 2 months before visit –Assessment activities, motivators, organizational structure –What’s worked/hasn’t, specific areas to address, expectations Schedule of meetings established Site visit Short report completed within 30 days –Specific recommendations made on moving forward Library evaluates process after final report received

Sample Site Visit Schedule Meet with University Librarian/Contact person Presentation on effective assessment –90 minutes to 2 hours with Q&A and discussion Group Meetings – with management/administrative group –assessment-related group (if formed –different departments –functional areas (e.g. info literacy) Wrap-up

ARL Project Preliminary Observations after Phase I Visits No “cookie-cutter” libraries; diverse organizational cultures Sustainability seen as resource issue Management information system seen as crucial underpinning Integrating data use effectively into library management Many internal statistics (non-ARL related) kept Few libraries allocate sufficient resources (staff and funding) for assessment or management information Facilities renovation as catalyst to good assessment practices Good assessment work being done but not published/ disseminated or coordinated Need for basic/practical training on assessment methods/analysis Little understanding of what’s happening outside the library in data warehousing/management

Phase I Feedback from Participants One day is too short More resource materials would be helpful More examples presented Follow-up activity would maintain momentum Establishing an assessment-related “community” of practitioners would assist ongoing communication and development

Phase II Changes Site visit time increased from 1 day to 1 1/2 days Appropriate resource materials provided Consulting on a follow-up activity –Assessment plan implementation –Specific assessment effort Individual meeting at a professional conference Group meeting at a professional conference Stronger project Web presence

Phase II Participants Confirmed –Cornell –Emory –Kansas –Louisville –Pennsylvania State –Southern California –Texas Tech –Wayne State –Western Ontario Willing to commit (not confirmed) –Boston Public Library –Colorado State –Nebraska –Purdue Interest pending (maybe) –Alabama –Alberta –Marquette

Future Reporting Upcoming Presentations –Northumbria International Conference on Performance Measurement, Durham, England, August –3 rd International Evidence-Based Librarianship Conference, Brisbane, Australia, October –ARL Membership Meeting, Washington, D.C., October or in May 2006 Making Library Assessment Work Web Site – Final Report late summer 2006 ARL Newsletter Conference in Fall 2006?