1. Developing the DigiQUAL Protocol for Digital Library Evaluation
Martha Kyrillidou, Director, Statistics & Measurement Program
Sarah Giersch, Consultant
Association of Research Libraries
ACM / IEEE 2005 Joint Conference on Digital Libraries
Denver, Colorado, USA
June 9, 2005

2. Terminology & Context It’s…? “familiar” and digital
National Science Digital Library - Evaluating the entity and Evaluating the projects
DigiQUAL: measure service quality in DLs

3. Developing DigiQUAL Survey Items
Background: ServQUAL ->LibQUAL ->DigiQUAL
LibQUAL dimensions of library service quality [would like to use a radar chart instead of text if there is space]
DigiQUAL: 12 themes of DL service quality
Accessibility
Collection building
DL as community for users
Copyright
Navigability
Resource use
...developers
Role of federations
Interoperability
Evaluating collections
...reviewers
DL sustainability

4. Pilot Testing Survey Items & Implementation
Your Digital Library?

5. Building a Survey Review and select items
Issue: aligning items to individual DL needs and users – vocabulary & content

6. Building a Survey (cont.)
Customize Survey
Issue: Flexibility vs. Standardization

7. Implementing Survey – Notification Methods
Links on site
Newsletters
Next to resources
Issues: no pop-ups, no individual s

8. Implementing Survey - Incentives
Issues: must be easily transferable, requires address - clear IRB

9. Analysis & Reporting - Issues
Interpreting results from individual digital libraries in the context of other sites
Sites reluctant to share data and results - possible negative impact with users and funding agencies

10. Outstanding Issues & Challenges
Unique DLs: niche market, critical mass, both?
Balance:
custom vs. generic content ->results
flexible vs. standard implementation -> scaling
Mixed methods
Preserving user privacy
Collecting truly useful data
Moving target: digital libraries as… it depends

11. Next Steps Continue pilot testing: NSDL and beyond
Expand implementation
Consider sustainability models
Want to Participate? Contact:
Sarah Giersch
Martha Kyrillidou
For more info: