Assessment for Impact: Turning Data into Tangible Results Laura Miller Tim Morton Paul Rittelmeyer Library Assessment Conference Seattle, WA August 5, 2008

Balanced Scorecard LibQUAL+ Surveys Users Staff Focus Groups Follow-up Interviews Statistics (circ, ILL, gate counts, purchase requests, etc.) Where Does Data Come From?

Improve Library Services Inform Collections Decisions Influence Satisfaction in the Workplace Support Budget Requests Contribute to Staff Development What Does Data Do?

Metric U.4.b. Turnaround time for new book and ILL requests (external resources). Target1: Satisfy the turnaround targets 75% of the time. Target2: Satisfy the turnaround targets 50% of the time. Method: New Books: Fill requests in 7 days. A sample of in-print US titles requested by University-affiliated patrons will be drawn from the request database by Management Information Services. Turnaround time from patron request to availability will be measured. ILL: Fill requests in 7 days. Turnaround time will be measured by ILS from patron request to notification that the book or article is available. Improve Library Service Rush Order (aka Purchase Express)

Improve Library Service Acquisitions (order to receipt) 2-4 days AUL OK

Improve Library Service Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day AUL OK

Improve Library Service Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day AUL OK Balanced Scorecard

Improve Library Service Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day Patron request/selector review/placing order ? AUL OK Balanced Scorecard

Improve Library Service Acquisitions (order to receipt) 2-4 days Cataloging (receipt to shelf prep to notifying patron) 1 day Patron request/selector review/placing order ? AUL OK Balanced Scorecard Selector training Back ups for /order placement Funding for expedited shipping

Inform Collections Decisions LibQUAL+ and satisfaction with journals

Inform Collections Decisions LibQUAL+ and satisfaction with journals

Focused on low-scoring areas Approached a diverse group of faculty Asked for specific needs, wants and suggestions Kept the interviews brief Targeted Interviews Following up with faculty Inform Collections Decisions

Dissatisfaction was not caused by single factor – no smoking gun Perceived access was a problem – in one area we already subscribed to all but one of the requested journals Lack of foreign titles Missing or incomplete backfiles What We Learned? Inform Collections Decisions

To address access, we instituted a major, ongoing re-design of our website We invested in a electronic resource management system (Serials Solutions 360) We prioritized the purchase of science backfiles (Wiley, Elsevier) and new publisher packages In the arts, we put significant funds into the purchase of quality images, or, if they could not be bought, we digitized them ourselves The Results Inform Collections Decisions

Influence Staff Satisfaction Library Worklife Survey Conducted in even-numbered years Available to all staff via online form with reminders Response Rates: 2004 – 60%, 2006 – 52.2% Two-Step Analysis/Response Process 1.Online survey Concerns identified 2.Focus groups, one-on-one meetings What can be done to address concerns?

Influence Staff Satisfaction Problems IdentifiedSolutions Implemented Staff dont know about other departments Open Houses for departments Staff dont know whats going on system-wide Did You Know?, Job/Position Announcements Administration not available to hear concerns Brown Bag Lunches, Open Door Policies Impacts Seen Change Library staff in other departments are familiar with what I do. 28.3% 17.3% 11.0% The Library administration is willing and available to listen to 52.3% 37.3% 15.0% my concerns, comments, and recommendations. The Library administration effectively communicates to staff.36.9% 36.6% 0.3% Collaboration is encouraged and supported in my job. 69.0% 75.8% -6.8% I get cooperation from other departments when we work together. 71.2% 74.0% -2.8%

Influence Staff Satisfaction Problems IdentifiedSolutions Implemented Need more staff and better distribution of work 24 new positions created Inadequate facilities, technology and support resources Library IT involved with allocations Facilities staff increased Balanced scorecard metric for facilities improvement Impacts Seen Change I have adequate resources and materials to complete assignments. 59.5% 49.2% 10.3% I feel safe in my work environment. 87.9% 81.5% 6.4% The Library is concerned about and addresses my ergonomic needs67.0% 76.3% -9.3% I am physically comfortable in my work environment (temperature, light, noise). 56.4% 44.6% 11.8%

Influence Staff Satisfaction Problems IdentifiedSolutions Implemented Like Administration, HR not always available New HR staff hired HR moves from Admin area into library Salaries are inadequate Extra allocations towards salary adjustments No opportunity for growth Better advertising of existing policies Mentoring program Impacts Seen Change My salary and benefits are reasonable for the work I do. 43.9% 21.5% 22.4% The salary I receive is equitable when compared to co-workers in similar grade levels or ranks. 49.0% 30.9% 18.1% My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5% There are opportunities for me to advance at the Library. 36.8% 28.1% 8.7% There are adequate staff development opportunities provided by the Library. 64.2% 57.3% 6.9%

Influence Staff Satisfaction The Overall Impact Change My work at the Library helps me achieve my career goals. 55.4% 35.9% 19.5% I am satisfied with my job. 63.2% 58.5% 4.7% Overall Job Satisfaction Category65.5% 58.0% 7.5%

Support Budget Requests 80% of Music faculty responded to the 2004 Faculty Survey. Of these respondents, 88% chose sound recordings as their highest priority, making it the highest priority item for the department. While every Music faculty respondent rated the resource sound recordings on a 1-5 scale, the category received one of the lower ratings for satisfaction, a The rating for sound recordings among faculty as a whole was 3.89.

Support Budget Requests Music Library Budget

Raising awareness of traditional practices as assessment tools (ref counts, gate counts, circ data, ILL data) Need to review, revise, reassess our BSC metrics and other tools. Assessment is ongoing. Managers and all staff need to learn how to better use data and assessment results in daily operations. Continue to move assessment out of MIS only. Position MIS as consultant/coordinator or assessment done throughout the organization. Assessment is for everyone! Challenges Ahead Promoting a Culture of Assessment