But What Does It Mean ? Using Statistical Data for Decision Making in Academic Libraries Steve Hiller University of Washington Libraries Seattle, Washington USA

North American Academic Libraries Swimming (or is it drowning?) in data –Electronic resources/collections use data –User assessment/survey data –Performance measures/outcomes based data ARL New Measures Initiative User-Centered Libraries/Culture of Assessment (organizational refocus)

Assessment Needs DLF report 2002 Collect meaningful, purposeful data Develop skills to gather, analyze, interpret, present and use data Develop comprehensive assessment plans Organize assessment as a core activity Compile and management assessment data Understand trends

DLF Report – Bleak Outlook The results of the DLF study suggest that individually, libraries in many cases are collecting data without having the will, organizational capacity, or interest to interpret and use the data effectively in library planning. Denise Troll Covey (Author)

But Wait! There is Optimism! Value of data recognized Data definition and collection more standardized Performance measures better articulated National impetus in higher education and libraries Organizational structure change;strategic planning MIS/Assessment positions growing in libraries Success stories growing

Obstacles to Using Statistical Data Too much data Not the right type of data Timeliness Library organizational structure Statistical skills and understanding Innumeracy

STATISTICAL LITERACY? Everyone knows that you can use statistics to prove anything. 14% of all people know that. Homer Simpson

The Data Think critically: How and why were they generated? Where do the numbers come from? What do they represent? Can you compare them? To what? Do they make sense? Can we use data to improve performance?

Communicating Results Keep it Simple: Less is More Provide executive summary or equivalent Briefly cover scope and methodology Identify key findings (not all results) Mix text, data, and graphics Avoid jargon Make it understandable to your audience(s) Identify action items or follow-up

Presentation for Management Presentation is different for management than for research publication. Highlight: –Whats important –How the information can be used to improve services –Ways to communicate externally and internally –Recommendations for handoffs

Be Graphic!!! Often the most effective way to describe, explore and summarize a set of numbers – even a very large set – is to look at pictures of those numbers. Furthermore, of all methods for analyzing and communicating statistical information, well designed data graphics are usually the simplest and at the same time the most powerful. Edwin Tufte The Visual Display of Quantitative Information

Edward Tufte "The Leonardo da Vinci of data." THE NEW YORK TIMES

The Visual Display of Quantitative Information (new second edition) Envisioning Information Visual Explanations Edward Tufte "The Leonardo da Vinci of data." THE NEW YORK TIMES

University of Washington Libraries Assessment Large scale user surveys done triennially since 1992 In-library use surveys done triennially since 1993 Annual topic for focus group sessions since 1998 Observation and usability studies on computer use E-metrics in online usage Process improvement in technical services

UW Libraries Assessment Communication Within 2-4 weeks after data receipt: –Initial data and analysis posted internally –Presentations made to library management Within 4-8 weeks after getting results: –Data and analysis posted on public Web site –Presentations made to other library groups After 8 weeks: –Written reports for local publication –Presentations made to external groups

UW Libraries Assessment Web Site Faculty and Student Surveys

Library Directions Article

Triennial Survey – Response Rate 2001 Sent 2001 Returned Faculty %40%31%28% Grad %46%41%56% Undergrad (2002) %39%23%41%

LIBRARY DISSATISFACTION IS LOW Percent Not Satisfied (Those marking 1 or 2 on scale of 1 to 5)

Frequency of In-Person Visits and Remote Access Uses 1998/2001 (% who use at least weekly)

Faculty Library Use Pattern 2001 by Academic Area (Those who use libraries at least weekly)

UWAnnual Loans and In-Library Use 1995 to 2001

ARL Total Circulation (Mean/Median)

Using Statistical Data in Management Not! Many DLF respondents reported surveys whose results were never analyzed... Libraries appear to be slow in acquiring the skills needed to use survey data. (Troll, 2002) Barriers to using LibQUAL+ data by participants: A dearth of in-house statistical skills, a lack of organizational culture that encourages assessment, low sample sizes, lack of time and money to work with results, need for more documentation to accompany the data. (Waller and Hipps, 2002)

Using Statistical Data in Management Yes! Library uses data in decision making Staff have data analysis skills Data is timely Resources are available to effect change Communicate results/action in timely manner Validate with complementary information Organization primed for action

University of Washington Libraries Data Use and Action Facilities planning and operation Open hours Access to online information Service improvements Recognizing a diverse user community

Reasons for Visiting Libraries 2001 (Among those visiting at least weekly)

Facilities Action Libraries are student places Provide diversified study and work areas Add computers (both library and those with application software) Extend hours of opening (but not reference) Increase user space; decrease collections space

User Priorities for Library are Online Access Top Priorities Identified in 2001 Library PriorityFacultyGraduate Students Undergraduates Full-text to desktop 73.5%72.8%54.9% Online access to older journals 59.6%61.8%40.0% Maintain quality of print collection 57.4%53.3%24.5% Electronic reserves23.8%37.5%49.5%

Priorities Differ by Academic Area 2001

Electronic Information Action Cancel print subscriptions if electronic available in sciences and health sciences New subscriptions are electronic only, if available Consortial purchases for more ejournal content Purchase electronic backfiles for older journals Provide access to full-text aggregator products Purchase online editions of reference works

Conclusion Use of Statistical Data in Library Management is Growing in North American Research Libraries Further progress needed: –Standardized data definitions and collections methods –Library staff expertise in data analysis and presentation –Organizational culture that facilitates change –Institutional planning and evaluation process that establishes goals and priorities that are measurable