Usage & Usability Denise A. Troll Distinguished Fellow, Digital Library Federation Associate University Librarian, Carnegie Mellon June 16, 2001 – LRRT, ALA

Digital Library Federation Initiative on usage, usability, & user support Distinguished Fellowship Convene library directors to outline research into how & why libraries are changing Conduct a usage & usability survey to discover DLF library issues & practices

Library Trends Traditional Inputs & Outputs Digital Inputs & Outputs What other changes are occurring in libraries? What do we know about the context? What’s really happening & why?

How & Why Are Libraries Changing? Motivational white paper Library Director Meeting Read & comment on the white paper Submit the 5 most important reasons for tracking trends, the audiences, & the indicators Expected consensus on 5-6 new measures

Outsell to Survey Academics Purpose of survey Study use of internal & external information Examine relevant environmental factors Identify gaps Provide trend data Random sample > 3000 Conduct survey fall 2001 Raise $$

Usage & Usability Survey ContactedParticipatedRate Institutions262492% Individuals837186% What data do DLF sites gather? What do they do with the data? What works well? What doesn’t work well? Creating a culture of assessment

Data Gathered Traditional & digital INPUTS Traditional & digital OUTPUTS Traditional & digital OUTCOMES User satisfaction ratings Service quality gaps Usability Referral URLs What about cost-effectiveness & learning & research outcomes?

Data Struggles Gathering the right data Gathering comparable data Interpreting the data Compiling & managing the data Figuring out how to use the data How do we do this cost-effectively?

Reference RefQuick RefReal RefDirectTechRefer Ref Desk In person Ref Desk Telephone Librarian Desk Telephone Chat Other What data to gather? For what purpose? How often? At what cost? Compare with what? What about user demographics? What about user privacy? Can chat transcripts, , & FAQ be used to create a reference database? What about turn-around time?

Should we operate both print & e-reserves? How do you define cost-effectiveness? E-Reserves Costs increase Technology Staff Staff training Disk space Revenues decrease Photocopying Fines

Implications of Assessments Reorganizing, recruiting, & creating positions Usability & Interface Specialist Director of Organizational Effectiveness Distance Education Department Training staff to gather, use, & present data effectively Stretching the budget

Creative Use of Web Statistics Based on # page hits Put links to high-use pages high in the hierarchy Keep high-use pages complete & up-to-date Put links to low-use pages low in the hierarchy Publicize or reduce maintenance of low-use pages Frequent queries Put links to queried pages high in the hierarchy Revise site vocabulary

Creative Use of Web Statistics Use by domain, location, or user group Facilitates collection & service development Indicates user satisfaction Helps target publicity Referral URLs indicate satisfaction & impact Do publicity or redesign increase use? Low use times are for system maintenance

Survey Questionnaires Why conduct? – To assess usage patterns; shifts in attitude; user needs, expectations, & satisfaction; service quality; market niche; & collection strengths, weaknesses, & relevance to the curriculum How apply the results? – Inform budget requests & investments; justify expenditures; reallocate resources; set priorities; target training & instruction

Focus Groups Why conduct? – To understand what users do or want to do; assess the use, effectiveness, & usefulness of collections & services; verify the results of previous research; inform new research How apply the results? – Confirm hypotheses, assumptions, & user needs; plan library renovations, training, instruction, publicity; inform collection development & access decisions; inform or validate interface design plans; reorganize the library

Think-Aloud Protocols Why conduct? – To identify problems in interface design, functionality, navigation, & vocabulary How apply the results? – Inform redesign of the library web site & digital collections; inform customization of the OPAC interface; verify improvements in service quality & interface design; develop library instruction; revise metadata schemes

Card sorting exercises – To group web site links, & label links & groups Heuristic evaluations – To identify problems in interface design, functionality, & navigation Paper prototypes & scenarios – (like protocols) Discount Usability Research Web Site Vocabulary

Serious Issues Lack of standards, guidelines, benchmarks, comparable data, & articulated proficiencies Recruiting representative human subjects Bias & bureaucracy Redundancy Interpretation Application