1. Introduction to Digital Libraries Week 5: Repositories, Identifiers and the Kahn/Wilensky Framework
Old Dominion University
Department of Computer Science
CS 751/851 Fall 2006
Michael L. Nelson
9/27/06

2. Arms, Ch. 13 repositories archives
“any computer system whose primary function is to store digital material for use in a library”
a collection of “stuff”
archives
repositories that make longevity promises
covered in a future lectures

3. “Key Concepts in the Architecture of the Digital Library”
next 8 slides taken from Bill Arm’s seminal article in the inaugural issue of D-Lib Magazine:

4. The technical framework exists within a legal and social framework
DLs no longer represent systems specific to academics or information specialists
content influences how the DL is used
architecture must allow the implementation of various policies

5. Understanding of digital library concepts is hampered by terminology
“common English” != “professional English”
multiple professional jargons too
What do these words mean to you?
copy
publish
content
document
work

6. The underlying architecture should be separate from the content stored in the library
general purpose functions and content-specific functions should be separated
TL analogy:
the more specific the bookshelf is to holding actual books, the harder it is to repurpose the bookshelf in the future

7. Names and identifiers are the basic building block for the digital library
names != addresses
in any DL architecture diagram, (almost) anything that can be drawn can be named
consider the impact that handles/DOIs have had on the publishing/DL community

8. Digital library objects are more than collections of bits
objects = metadata + data
“but what is metadata?”
don’t ask hard questions
figure 2 in

9. The digital library object that is used is different from the stored object
what you store is not necessarily what you get
storage and dissemination are separate events, and can represent separate formats
also, potentially separate from the application-specific format

10. Users want intellectual works, not digital objects
The DL architect’s needs should not inconvenience the users’ needs
recombination of objects
what is an object in your world view?
figure 4 in

11. Repositories must look after the information they hold
“Repository Access Protocol”
Kahn Wilensky Framework
figure 3 in

12. A Framework for Distributed Digital Object Services
More commonly known as the Kahn/Wilensky Framework (KWF)
A high level document, not even detailed enough to be an architecture, that defines some of the key concepts and terms that form the basis for the next generation of DLs
DLs beyond “make the ftp server look nice”

13. Key KWF Terms digital objects (DOs) repository handles
a unit of exchange for the DL with a particular data structure and characteristics
repository
the place where DOs live
handles
a unique, persistent name for a DO

14. KWF Originator makes a Data which consists of Digital Object
which comes
from a handle
generator
Handle
which can go in a
Repository
which is accessed by
which registers the DO’s handle with a
Handle
Server
Repository
Access
Protocol
(RAP)
at which point the DO becomes
a registered DO

15. Digital Objects Digital object = data + key-metadata
data is typed; core types include:
bit-sequence / set-of-bit-sequences
digital-object / set-of-digital-objects
handle / set-of-handles
other types can be defined, and registered with a global type registry
definition and registration left undefined
similar to MIME?
key-metadata includes handle, possibly other metadata (left undefined in KWF)

16. Digital Objects Typed data; example from KWF: Composite DOs:
a DO subtype: computer-science-tech-report
with metadata: author, institution, series, etc.
Composite DOs:
a DO with data of type digital-object
non-composite DOs are elemental DOs
composite DOs can be used to collect similar works together
composite DO that contains a DO for each work of Shakespeare...

17. Changing Digital Objects
Mutable DOs can be changed once placed in a repository
key-metadata cannot be changed -- the DO’s handle does not change!
Immutable DOs cannot be changed once placed in a repository
however, it can be deleted

18. Uniform Resource Identifiers
URI
RFC 2396
RFC 1738
URL
RFC 2141
URN

19. URIs & URNs registered URI schemes registered URN namespaces
registered URN namespaces

20. From RFC 2396
“A URI can be further classified as a locator, a name, or both. The
term "Uniform Resource Locator" (URL) refers to the subset of URI
that identify resources via a representation of their primary access
mechanism (e.g., their network "location"), rather than identifying
the resource by name or by some other attribute(s) of that resource.
The term "Uniform Resource Name" (URN) refers to the subset of URI
that are required to remain globally unique and persistent even when
the resource ceases to exist or becomes unavailable.”

21. URLs
URLs are tightly coupled with the physical location of an object, and are thus more likely to be transient
“Error File not found”
Tricks to make URLs more durable:
plan ahead when constructing web site structure
use good DNS CNAMEs
symbolic links on filesystems
http server redirects

22. URNs
But with all the tricks available, URLs are not suitable for archival use in DLs
how long will this URL (a report in LTRS):
be good?
how to handle mirroring, replication, etc.?
“appropriate copy” problem…
mnemonic:
URL = IP address ( )
URN = IP name (blearg.cs.odu.edu)

23. Handles
Handles can be thought of as a Uniform Resource Name (URN) implementation
for historical comparison of efforts
contains info about the handle system
persistence
location independence
multiple instances
Handles are of the general form:
GlobalAuthority.LocalAuthority/LocallyUniqueString
or, for example:
NASA.LaRC/tm112871

24. NASA.LaRC/tm112871
“NASA” would be assigned from the global naming authority
“LaRC” would be created by who registered “NASA”, and the entire string “NASA.LaRC” would be registered
“tm112871” is a locally unique string generated by “LaRC”
ODU.CS/tm is possible...

25. Handle Syntax In URL-type syntax: Using a proxy server:
<a href=“hdl:NASA.LaRC/tm112871”>
“hdl” is a scheme; handle is resolved into a URL by locally defined handle server
see for a good list of schemes and naming projects
Using a proxy server:
<a href=“
hdl.handle.net performs resolution
from:

26. Handles
Observation: isn’t the handle system just the Domain Name System (DNS) all over again?
The need for URNs for just general WWW use is obvious; the need for them in DLs even more so...

27. Semantics in Names Two schools of thought:
semantic clues in names, such as:
NASA.LaRC/tm112871
are:
good: easy to parse, remember, map to real-world concepts, etc.
bad: names are not for human consumption, are hurtful or restrictive in the long run, etc.

28. “I Love Mom” (without Semantics)
image from Eddie Kohler

29. Purls Persistent URLs (Purls) examples: http://purl.net/, OCLC
Maps stable URLs (registered in purl.net space) to transient URLs (i.e. cs.odu.edu/~user/ space)
examples:

30. DOIs Digital Object Identifier System (DOIs) http://www.doi.org/
no semantics in the names (well, that’s not always true…)
driven by the publishing industry
examples:
doi: /september2002-rasmussen /
resolver:

31. info URI “info” URI proposal
how to identify “stuff” that does not resolve?
LCSHs?
XML namespaces?
“Truth”, “Love”, “Beauty”, etc.
how to promote locally unique ids to globally unique?
URNs require persistence…

32. Repositories
“A network accessible storage system in which digital objects may be stored for possible subsequent access or retrieval” (KWF)
A stored DO is a DO that resides in a repository
A registered DO is a DO that the repository has registered with a handle server
storing and registering can be the same or different processes

33. Repositories A repository keeps a properties record for each DO
contains key-metadata and any other metadata the repository chooses to keep
A repository of record (ROR) is the first repository that a DO is placed in
ROR authorizes additional instances of the DO
A dissemination is the result of an access service request

34. Repository Access Protocol (RAP)
“Protocol” may be misleading, its really just the skeleton for a protocol
RAP is designed to be simple
repositories themselves should be simple
KWF defines 3 basic operation classes:
ACCESS_DO
DEPOSIT_DO
ACCESS_REF
this is the catch-all operation for all meta-services...

35. RAP RAP is fleshed out more in Cornell CS 95-TR1540
Where KWF suggested that the operations would take “metadata”, “key-metadata”, and “digital object” as arguments, TR1540 splits some of those into separate operations
RAP could be implemented as a subset of a more sophisticated protocol (Dienst, Z39.50, etc.)
prelude to the Open Archives Initiative protocol for metadata Harvesting (OAI-PMH)

36. RAP

37. Terms and Conditions
First lengthy discussion with respect to KWF in Cornell CS 95 TR-1593
TC are attached to:
each DO
dissemination
repository
TC are a precondition for any operation on the above
Repositories responsible for enforcing TC

38. Why Are TC Difficult?
Wide open model -- “everyone can access and do everything” is much simpler
How do you:
specify TC?
inform user of TC?
negotiate TC?
enforce TC?
esp. with respect to 3rd party enforcers

39. KWF Now
The KWF was never “implemented” in a real DL , (the 1995 Cornell TRs notwithstanding), yet it has influenced all repository & object model projects that followed
e.g. Warwick Framework, Fedora, Buckets/SODA, Dienst, OAI-PMH
T&C, or “Rights Expressions”, have mostly been moved out of the DL/repository protocols and into complex object formats
Koyle, “Rights Expression Langages”, 2004

40. Objects vs. Archives
“Repositories must look after the information they hold”
This is the tenet that I question…
Most DL objects still bound to the applications that generate or render the objects