1. Usage & Usability Denise A. Troll Distinguished Fellow, Digital Library Federation Associate University Librarian, Carnegie Mellon June 16, 2001 – LRRT, ALA

2. Digital Library Federation Initiative on usage, usability, & user support http://www.clir.org/diglib/use/useframe.htm http://www.clir.org/diglib/use/useframe.htm Distinguished Fellowship Convene library directors to outline research into how & why libraries are changing Conduct a usage & usability survey to discover DLF library issues & practices

3. Library Trends Traditional Inputs & Outputs Digital Inputs & Outputs What other changes are occurring in libraries? What do we know about the context? What’s really happening & why?

4. How & Why Are Libraries Changing? Motivational white paper http://www.clir.org/diglib/use/whitepaper.htm http://www.clir.org/diglib/use/whitepaper.htm Library Director Meeting Read & comment on the white paper Submit the 5 most important reasons for tracking trends, the audiences, & the indicators Expected consensus on 5-6 new measures

5. Outsell to Survey Academics Purpose of survey Study use of internal & external information Examine relevant environmental factors Identify gaps Provide trend data Random sample > 3000 Conduct survey fall 2001 Raise $$

6. Usage & Usability Survey ContactedParticipatedRate Institutions262492% Individuals837186% What data do DLF sites gather? What do they do with the data? What works well? What doesn’t work well? Creating a culture of assessment

7. Data Gathered Traditional & digital INPUTS Traditional & digital OUTPUTS Traditional & digital OUTCOMES User satisfaction ratings Service quality gaps Usability Referral URLs What about cost-effectiveness & learning & research outcomes?

8. Data Struggles Gathering the right data Gathering comparable data Interpreting the data Compiling & managing the data Figuring out how to use the data How do we do this cost-effectively?

9. Reference RefQuick RefReal RefDirectTechRefer Ref Desk In person Ref Desk Telephone Librarian Desk Telephone Email Chat Other What data to gather? For what purpose? How often? At what cost? Compare with what? What about user demographics? What about user privacy? Can chat transcripts, email, & FAQ be used to create a reference database? What about turn-around time?

10. Should we operate both print & e-reserves? How do you define cost-effectiveness? E-Reserves Costs increase Technology Staff Staff training Disk space Revenues decrease Photocopying Fines

11. Implications of Assessments Reorganizing, recruiting, & creating positions Usability & Interface Specialist Director of Organizational Effectiveness Distance Education Department Training staff to gather, use, & present data effectively Stretching the budget

12. Creative Use of Web Statistics Based on # page hits Put links to high-use pages high in the hierarchy Keep high-use pages complete & up-to-date Put links to low-use pages low in the hierarchy Publicize or reduce maintenance of low-use pages Frequent queries Put links to queried pages high in the hierarchy Revise site vocabulary

13. Creative Use of Web Statistics Use by domain, location, or user group Facilitates collection & service development Indicates user satisfaction Helps target publicity Referral URLs indicate satisfaction & impact Do publicity or redesign increase use? Low use times are for system maintenance

14. Survey Questionnaires Why conduct? – To assess usage patterns; shifts in attitude; user needs, expectations, & satisfaction; service quality; market niche; & collection strengths, weaknesses, & relevance to the curriculum How apply the results? – Inform budget requests & investments; justify expenditures; reallocate resources; set priorities; target training & instruction

15. Focus Groups Why conduct? – To understand what users do or want to do; assess the use, effectiveness, & usefulness of collections & services; verify the results of previous research; inform new research How apply the results? – Confirm hypotheses, assumptions, & user needs; plan library renovations, training, instruction, publicity; inform collection development & access decisions; inform or validate interface design plans; reorganize the library

16. Think-Aloud Protocols Why conduct? – To identify problems in interface design, functionality, navigation, & vocabulary How apply the results? – Inform redesign of the library web site & digital collections; inform customization of the OPAC interface; verify improvements in service quality & interface design; develop library instruction; revise metadata schemes

17. Card sorting exercises – To group web site links, & label links & groups Heuristic evaluations – To identify problems in interface design, functionality, & navigation Paper prototypes & scenarios – (like protocols) Discount Usability Research Web Site Vocabulary

18. Serious Issues Lack of standards, guidelines, benchmarks, comparable data, & articulated proficiencies Recruiting representative human subjects Bias & bureaucracy Redundancy Interpretation Application