1. ECDL 2006, Alicante, September 18, 2006
What is a Successful
Digital Library?
ECDL 2006, Alicante, September 18, 2006
Rao Shen, Naga Srinivas Vemuri, Weiguo Fan, and Edward A. Fox

2. Acknowledgements (Selected)
Sponsors: NSF grant ITR , ASOR, CWRU, ETANA, Vanderbilt U., Virginia Tech
Faculty/Staff: Lillian Cassel, Debra Dudley, Manuel Perez, …
VT (Former) Students: Aaron Krowne, Ming Luo, Fernando Das Neves, Ricardo Torres, Hussein Suleman, …

3. Acknowledgements (Selected)
Karen Borstad, MPP
Giorgio Buccellati, UCLA
Douglas Clark, Walla Walla College
Joanne Eustis, CWRU
Nick Fischio, CWRU
Israel Finkelstein, Tel-Aviv University
Paul Gherman, Vanderbilt U.
Andrew Graham, U. Toronto
Tim Harrison, U. Toronto
Larry Herr, Canadian University College
Christopher Holland, LRP
Paul Jacobs, Mississippi State U.
Douglas Knight, Vanderbilt U.
Stan LaBianca, Andrews U.
David McCreery, Willamette U.
Eric Meyers, Duke U.
Adam Porter, Illinois College
Jack Sasson, Vanderbilt U.
Tom Schaub, Indiana U. of Penn.
Randall Younker, Andrews U.

5. ETANA-DL Website

6. ETANA-DL’s Member Collections

7. Outline Prior work DL success model Case study Conclusion
From end user perspective
Case study
Conclusion

8. Prior Work on Measuring DL Success
inspection of NCSTRL
Usability of DLs
has an example
evaluation of ACM, IEEE-CS,
NCSTRL, and NDLTD
evaluation of ADL
evaluation of ADEPT
Intention to re/use
predict
Technology
acceptance model
has an example
Venkatesh
system usage
has an indicator
DeLone et al.
IS success model
has an example
Seddon
Ellis
Information seeking
behavior model
has an example
Kuhlthau
DL quality model
has an example
Gonçalves

9. Behavioral Attitude & Intension to Use — Venkatesh Model of IT Adoption
Unified Theory of Acceptance and Use of Technology
performance expectancy
effort expectancy
intention to use system
system usage
social influence
facilitating conditions

10. Venkatesh Model of IT Adoption
Performance expectancy:
perceived usefulness, extrinsic motivation, job-fit, relative advantage, and outcome expectations
Effort expectancy
the degree of ease associated with the use of system
Social influence
Subjective norms, social factors, and image
Facilitating conditions
the degree to which an individual believes that an organizational and technical infrastructure exist to support the system

11. DeLone and McLean Model of IS Success
System
Quality
Use
Individual
Impact
Organization
Impact
Information
Quality
User
Satisfaction

12. Seddon Model of IS Success
Net Benefits to:
Individuals
Organizations
Society
System
Quality
Perceived
Usefulness
Information
Quality
User
Satisfactions

13. Outline Prior work DL success model Case study Conclusion
From end user perspective
Case study
Conclusion

14. DL Success Model 5S and minimal DL Synthesize
IS success and adoption models (see above)
Information life cycle model (Borgman et al.)
5S-based DL quality model (Gonçalves et al.)
Information-seeking behavior models (Ellis’ and Kuhlthau’s)
From end user perspective

15. Informal 5S & DL Definitions DLs are complex systems that
help satisfy info needs of users (societies)
provide info services (scenarios)
organize info in usable ways (structures)
present info in usable ways (spaces)
communicate info with users (streams)

16. 5S and DL formal definitions and compositions (April 2004 TOIS)

17. Information Life Cycle Borgman et al.: Workshop Report on
Thanks to Christine Borgman and others at this workshop.
Borgman et al.:
Workshop Report on
Social Aspects of
Digital Libraries:
ucla.edu/DL/

18. Information Life Cycle
Creation
Active
Authoring
Modifying
Social Context
Using
Creating
Organizing
Indexing
Retention
/ Mining
Accessing
Filtering
Storing
Retrieving
Semi-
Active
Utilization
This is a simplification of the previous slide.
Distributing
Networking
Inactive
Searching

19. describing organizing, indexing
completeness,
conformance
accuracy,
Active
accessibility, preservability
Semi-active
preservability,
describing organizing, indexing
authoring, modifying,
storing, archiving,
networking
similarity,
timeliness,
creation
distribution
searching,
browsing,
recommending
utilization
seeking
pertinence,
accessing,
filtering
Information life cycle with respective dimensions of quality added for each major phrase and related activities
significance,
relevance
Gonçalves et al.
accessibility,
Inactive
timeliness

21. describing organizing, indexing
E: Ellis’ model
K: Kuhlthau’s model
completeness,
conformance
accuracy,
Active
accessibility, preservability
Semi-active
preservability,
describing organizing, indexing
authoring, modifying,
storing, archiving,
networking
similarity,
timeliness,
creation
distribution
E1:starting
E2: chaining
E3: browsing
E4: differentiating
E5: monitoring
E6: extracting
initiation
K1:
selection
K2:
K3:
exploration
K4:
formulation
K5:
collection
presentation
K6:
utilization
seeking
Inactive
DL Success Constructs

22. DL Success Model relevance adequacy timeliness reliability
understandability
scope
information quality
(IQ)
information quality
(IQ)
performance
expectancy
(PE)
behavioral
Intention to
(re)use
satisfaction
system quality
(SQ)
social influence (SI)
accessibility
ease of use
joy of use
reliability
user
interface

23. DL Concepts Regarding Information
associated with
digital object
metadata specification
consist of
consist of
associated with
collection
metadata catalog
consist of
repository

24. Information Quality (IQ)
Digital Library IQ
accessibility
preservability
pertinence
relevance
similarity
significance
timeliness
digital object
Information Quality (IQ)
digital object
metadata specification
understandability
relevance
timeliness
reliability
adequacy
accuracy
completeness
conformance
metadata
specification
consistency
completeness
catalog
catalog
collection
impact factor
completeness
collection
adequacy
repository
consistency
completeness
repository
scope

25. DL Success Model relevance adequacy timeliness reliability
understandability
scope
information quality
(IQ)
performance
expectancy
(PE)
behavioral
Intention to
(re)use
satisfaction
system quality
(SQ)
system quality
(SQ)
social influence (SI)
accessibility
ease of use
joy of use
reliability
user
interface

26. Digital Library SQ System Quality (SQ) service composability
reusability
extensibility
efficiency
effectiveness
reliability
service
accessibility
reliability
ease of use
joy of use
performance
expectancy
user interface
screen design
navigation

27. Outline Prior work DL success model Case study Conclusion
From end user perspective
Case study
Conclusion

28. Case Study Part of requirements analysis for ETANA-DL
interviews with 5 prestigious archaeologists
Face to face workplace interviews with 11 archaeologists
Associate the 4 constructs of DL success model with the activities occurring in the seeking and utilization phases

29. DL Success Constructs Associated with Seeking and Utilization Phases
seeking phase
utilization phase
starting (E1/K1)
selection exploration
(E2-E6)/(K2-K3)
formulation (K4)
collection (K5)
presentation (K6)
social influence
DL visibility
information quality
adequacy, scope
accuracy
system quality
ease of use
joy of use (interface)
accessibility
performance
expectancy
usefulness
(interface)

30. DL Success Constructs Associated with Seeking Phase
E1: Starting’ activity in Ellis’ model (K1: ‘initiation’ stage in Kuhlthau’s model)
Social Influence (SI) — DL visibility
Publicize existence of a DL
Provide a DL alert service

31. DL Success Constructs Associated with Seeking Phase
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model)
Information Quality (IQ)
System Quality (SQ)
Performance Expectancy (PE)

32. DL Success Constructs Associated with Seeking Phase
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model)
Information Quality (IQ)
Adequacy (degree of sufficiency and completeness) of DL collections and metadata catalogs
Scope of DL repository

33. DL Success Constructs Associated with Seeking Phases
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model)
System Quality (SQ)
Ease of use
Joy of use

34. DL Success Constructs Associated with Seeking Phases
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model)
Performance Expectancy (PE)
Usefulness

35. DL Success Constructs Associated with Seeking Phases
E2-E6: ‘chaining’, ‘browsing’, ‘differentiating’, ‘monitoring’, and ‘extracting’ in Ellis’ model (K2-K3: ‘selection’ and ‘exploration’ stages in Kuhlthau’s model)
System Quality & Performance Expectancy
DL interface: screen design & navigation

36. DL Success Constructs Associated with Utilization Phase
K4-K6: ‘formulation’, ‘collection’, and ‘presentation’ stage in Kuhlthau’s model
Information Quality
information accuracy
information accessibility

37. Outline Prior work DL success model Case study Conclusion
From end user perspective
Case study
Conclusion

38. Conclusion
Lay the foundation for defining success of DLs from the view of DL end users
Assume a multi-theoretical perspective
Synthesize many related research areas in terms of theory and empirical work
Explicate and illustrate our approach by a case study with ETANA and usability
Connect with other work on DL quality: led by Emory funded by IMLS, DELOS …

39. See http://fox.cs.vt.edu/talks/2006/ 20060918ECDLsuccess.ppt
Questions? Comments?
See ECDLsuccess.ppt
Thank You!