Computers in Libraries 2003, Washington, DC Digital Library Collections & Services Landscape & Strategies Roy Tennant California Digital Library

Computers in Libraries 2003, Washington, DC Questions, Questions, Questions You will leave with more questions than answersYou will leave with more questions than answers If I do my job right, they will be the right questionsIf I do my job right, they will be the right questions Feel free to ask questions as we go alongFeel free to ask questions as we go along

Computers in Libraries 2003, Washington, DC “It was the best of times, it was the worst of times…” — A Tale of Two Cities, Charles Dickens

Computers in Libraries 2003, Washington, DC The Common Perception

Computers in Libraries 2003, Washington, DC The Reality Too many information sources with which to copeToo many information sources with which to cope A lack of human assistance at the time and place neededA lack of human assistance at the time and place needed Not enough ways to filter, sort, and otherwise narrow in on what is neededNot enough ways to filter, sort, and otherwise narrow in on what is needed Access is limited to what is free, or what has been purchased or rented on behalf of a clienteleAccess is limited to what is free, or what has been purchased or rented on behalf of a clientele Many useful resources are only available in printMany useful resources are only available in print

Computers in Libraries 2003, Washington, DC The Most Commonly Proposed Solution

Computers in Libraries 2003, Washington, DC Digital Library Myths Having everything in digital form will solve our information access problemsHaving everything in digital form will solve our information access problems Soon (or eventually) everything will be digitalSoon (or eventually) everything will be digital Any collection of digital objects can be a digital libraryAny collection of digital objects can be a digital library Everyone agrees about what comprises a “digital library” and how to build one — all we must do is do the workEveryone agrees about what comprises a “digital library” and how to build one — all we must do is do the work

Computers in Libraries 2003, Washington, DC A Digital Library Is… A collection of digital objects and/or information that is:A collection of digital objects and/or information that is: –Selected –Organized –Made Accessible –Preserved A set of services that help you to find and use those objects and informationA set of services that help you to find and use those objects and information Often supported by a physical collection and always by professional staffOften supported by a physical collection and always by professional staff

Computers in Libraries 2003, Washington, DC Outline Themes:Themes: –Library Catalogs –Digital Library Collections –Digital Reference –Libraries as Publishers –Cross-Database Searching Technologies:Technologies: –XML –Metadata –Interoperability

Computers in Libraries 2003, Washington, DC Library Catalogs We seem to be unable to provide an easy and effective information locating toolWe seem to be unable to provide an easy and effective information locating tool Keep in mind that only librarians like to search, everyone else likes to findKeep in mind that only librarians like to search, everyone else likes to find We are even failing at things we have explicitly tried to doWe are even failing at things we have explicitly tried to do Let’s take a look at the evidence…Let’s take a look at the evidence…

Computers in Libraries 2003, Washington, DC

Typical Searches Known ItemKnown Item “A Few Good Things”“A Few Good Things” ComprehensiveComprehensive

Computers in Libraries 2003, Washington, DC Typical Searches: Known Item The good: searches can be limited to a particular field: author, title, etc.The good: searches can be limited to a particular field: author, title, etc. The bad: limiting to a particular field doesn’t always act the way you expectThe bad: limiting to a particular field doesn’t always act the way you expect

Computers in Libraries 2003, Washington, DC

Typical Searches: “A Few Good Things” The one type of search we have so far ignored in library system designThe one type of search we have so far ignored in library system design A type of search that we can do something about todayA type of search that we can do something about today Bring Google-style relevance to library catalogs (e.g., for union catalogs, sort by number of holding libraries)Bring Google-style relevance to library catalogs (e.g., for union catalogs, sort by number of holding libraries)

Computers in Libraries 2003, Washington, DC Typical Searches: Comprehensive Most library catalogs hide many things available via regional cooperative or ILLMost library catalogs hide many things available via regional cooperative or ILL It is difficult, if not impossible, to search all appropriate journal databasesIt is difficult, if not impossible, to search all appropriate journal databases Most libraries do not provide good access to gray literature and web sitesMost libraries do not provide good access to gray literature and web sites Subject headings are often unintuitive, and catalogs give no guidanceSubject headings are often unintuitive, and catalogs give no guidance Catalogs give no chapter-level access to book contentCatalogs give no chapter-level access to book content

Computers in Libraries 2003, Washington, DC

The Rescue of Print Many library users want only that which is convenient (read digital)Many library users want only that which is convenient (read digital) Print resources are, therefore, increasingly overlooked (I call this the “convenience catastrophe”)Print resources are, therefore, increasingly overlooked (I call this the “convenience catastrophe”) We must fight this trend by enriching our catalog records with tables of contents, indexes, book covers, etc. to entice users to print booksWe must fight this trend by enriching our catalog records with tables of contents, indexes, book covers, etc. to entice users to print books

Computers in Libraries 2003, Washington, DC Digital Library Collections Applying the digital versions of traditional library activities (selecting, acquiring, etc.) to put library collections onlineApplying the digital versions of traditional library activities (selecting, acquiring, etc.) to put library collections online One of the most well-understood of digital library activitiesOne of the most well-understood of digital library activities Many examples, resources, consultants, and vendors are available to help a library digitize their collectionsMany examples, resources, consultants, and vendors are available to help a library digitize their collections

Computers in Libraries 2003, Washington, DC

Digital Reference Putting the human help where it’s needed — onlinePutting the human help where it’s needed — online Software is now available that provides for:Software is now available that provides for: –Queuing of patrons with audible alerts –Chat between librarian and user –Push web pages –Form sharing –Highlighting –Saved and/or ed transcripts –Statistics

Computers in Libraries 2003, Washington, DC

XML A method of creating and using tags to identify the structure and contents of a document — not how it should be displayedA method of creating and using tags to identify the structure and contents of a document — not how it should be displayed The tags used can be arbitrary or can come from a specificationThe tags used can be arbitrary or can come from a specification There are two types: well formed and valid…There are two types: well formed and valid…

Computers in Libraries 2003, Washington, DC Well-Formed XML Follows general tagging rules:Follows general tagging rules: –All tags begin and end But can be minimized if empty: But can be minimized if empty: –All tags are properly nested: Mark Twain Mark Twain –All attribute values are quoted: Music Music Software can check to make sure a given document follows these basic rulesSoftware can check to make sure a given document follows these basic rules

Computers in Libraries 2003, Washington, DC Valid XML Uses only specific tags and rules as codified by one of:Uses only specific tags and rules as codified by one of: –A document type definition (DTD) –A schema definition Only the tags listed by the schema or DTD can be usedOnly the tags listed by the schema or DTD can be used Software can take a DTD or schema and verify that a given document adheres to the rulesSoftware can take a DTD or schema and verify that a given document adheres to the rules Editing software can prevent an author from using anything except allowed tagsEditing software can prevent an author from using anything except allowed tags

Computers in Libraries 2003, Washington, DC Transforming XML XML Stylesheet Language — Transformations (XSLT)XML Stylesheet Language — Transformations (XSLT) –A markup language and programming syntax for processing XML –Used to transform XML to another format (e.g., to HTML for delivery to standard web clients) or from one set of tags to another An XML parserAn XML parser A method to bring all the pieces together if serving to the web (e.g., CGI program, Java servlet, etc.)A method to bring all the pieces together if serving to the web (e.g., CGI program, Java servlet, etc.)

Computers in Libraries 2003, Washington, DC Metadata Structured information about an object or collection of objectsStructured information about an object or collection of objects Types:Types: –Descriptive –Administrative –Structural –Preservation

Computers in Libraries 2003, Washington, DC Metadata Standards Dublin Core: a set of basic fields primarily for systems interoperabilityDublin Core: a set of basic fields primarily for systems interoperability MODS: a MARC-like bibliographic formatMODS: a MARC-like bibliographic format METS: a structural standard for encapsulating a digital object or set of digital objects, including one or more segments of descriptive and/or administrative metadataMETS: a structural standard for encapsulating a digital object or set of digital objects, including one or more segments of descriptive and/or administrative metadata

Computers in Libraries 2003, Washington, DC Libraries as Publishers Libraries are increasingly becoming involved with publishing activitiesLibraries are increasingly becoming involved with publishing activities University libraries are capturing scholarship before it leaves campus, and making it freely available to allUniversity libraries are capturing scholarship before it leaves campus, and making it freely available to all Two examples:Two examples: –Repositories –Book publishing

Computers in Libraries 2003, Washington, DC Repositories Two flavors:Two flavors: –Institutional –Topic or type focused Characteristics:Characteristics: –Often author-maintained; therefore metadata may be of uneven quality/quantity –Often compliant with the Open Archives Initiative harvesting protocol Benefits:Benefits: –Captures a grey literature not always collected by libraries –If OAI-compliant, can be “crawled” and indexed

Computers in Libraries 2003, Washington, DC

Dspace screen shot

Computers in Libraries 2003, Washington, DC

Aggregation Portal Subject Portal The OAI Model MIT UC Stanford

Computers in Libraries 2003, Washington, DC

Book Publishing Academic libraries and university presses are teaming up:Academic libraries and university presses are teaming up: –Libraries provide technical expertise, online access, persistence, professional collection management –University presses provide editing, print publication, imprimatur, marketing Case Study: University of California Press and the California Digital LibraryCase Study: University of California Press and the California Digital Library

Computers in Libraries 2003, Washington, DC

Book encoded in XML Information Transformation Web Server XSLT Stylesheet Presentation XHTML Document (no display markup)* HTML Stylesheet (CSS) * Dynamic document

Computers in Libraries 2003, Washington, DC

Other Metadata Library Catalog METS Repository MODS record DC record Structure Record Creation Program

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index Search Index Image metadata

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index Query & Profile

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index XSLT Results in XML

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index XSLT Results in XML Object request

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index XSLT Results in XML XSLT METS record in XML

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index XSLT Results in XML Book segment requested

Other Metadata Library Catalog METS Repository Project Profile Field Extraction Program MODS record DC record Structure Indexing Software Record Creation Program Search Index XSLT Results in XML Segment extracted and transformed to HTML XSLT

Computers in Libraries 2003, Washington, DC XML & XSLT Demonstration

Computers in Libraries 2003, Washington, DC Cross-Database Searching Necessary for the user to:Necessary for the user to: –Not have to know which database to search –Not have to go to each database individually to search –Not have to know the particular commands to search in each database An incredibly complex problem that will likely take years to come close to solving wellAn incredibly complex problem that will likely take years to come close to solving well

Computers in Libraries 2003, Washington, DC A Future I Would Like to See

Computers in Libraries 2003, Washington, DC A Model for Achieving It Repositories Web Search Engines Library Catalogs Journal Article Databases Digital Library Collections The Integration Engine The User Interface Online Assistance

Source: ARL Statistics

Computers in Libraries 2003, Washington, DC Interoperability The digital library “holy grail”The digital library “holy grail” Main requirement: widespread adoption of specific standards and protocolsMain requirement: widespread adoption of specific standards and protocols Progress:Progress: –XML as the basic syntax –OAI provides a harvesting model –METS, MODS, and DC are key metadata standards –Technologies such as Web Services provide real-time interoperability

Computers in Libraries 2003, Washington, DC Understanding the Landscape We must provide access to more resources than ever beforeWe must provide access to more resources than ever before Many are digital, some are notMany are digital, some are not Some are interoperable, many are notSome are interoperable, many are not We need to find ways to build unified user services from a disparate collection of resourcesWe need to find ways to build unified user services from a disparate collection of resources Tools and strategies for doing this are becoming availableTools and strategies for doing this are becoming available

Computers in Libraries 2003, Washington, DC Trends More and faster change (“change is the only constant”)More and faster change (“change is the only constant”) Better control of, and access to, historically difficult to access materials (e.g., working papers, data sets, etc.)Better control of, and access to, historically difficult to access materials (e.g., working papers, data sets, etc.) More publication options:More publication options: –Digital repositories –Institution-based peer-reviewed publication avenues (journals, books, etc.) A greater diversity of material types:A greater diversity of material types: –Multimedia, data, etc.

Computers in Libraries 2003, Washington, DC A Few Technologies and Trends to Watch Repository systemsRepository systems Systems for online peer review and publicationSystems for online peer review and publication New kinds of “cataloging” (e.g., Dublin Core, METS, MODS)New kinds of “cataloging” (e.g., Dublin Core, METS, MODS) XMLXML Open Archives InitiativeOpen Archives Initiative Web servicesWeb services Cross-database search toolsCross-database search tools

Computers in Libraries 2003, Washington, DC The Right Questions to Ask What do we need to do to serve our users better?What do we need to do to serve our users better? How can we build an infrastructure that can be used for a variety of purposes?How can we build an infrastructure that can be used for a variety of purposes? How can we better integrate access to print and digital material?How can we better integrate access to print and digital material? How can we interoperate with other systems and services?How can we interoperate with other systems and services? What should we stop doing so we can do what is more important?What should we stop doing so we can do what is more important?