1. Digital Libraries: Extending and Applying Library and Information Science and Technology CIKM 2000 November 9, 2000 Edward A. Fox fox@vt.edu http://fox.cs.vt.edu CS DLRL Internet TIC Virginia Tech, Blacksburg, VA, USA

2. Acknowledgements (Selected) F Mentors: JCR Licklider, Michael Kessler, Gerard Salton F Sponsors: Adobe, IBM, Microsoft, NLM, NSF, OCLC, SOLINET, SURA, UNESCO, US Dept. of Ed. (FIPSE), … F VT Faculty/Staff: Tony Atkins, Thomas Dunbar, Debra Dudley, John Eaton, Gwen Ewing, Peter Haggerty, Gary Hooper, Gail McMillan, Len Peters, James Powell, …  VT Students: Emilio Arce, Fernando Das Neves, Brian DeVane, Robert France, Marcos Goncalves, Scott Guyer, Robert Hall, Neill Kipp, Paul Mather, Tim McGonigle, Todd Miller, Constantinos Phanouriou, William Schweiker, Ohm Sornil, Hussein Suleman, Patrick Van Metre, Laura Weiss, …

3. Internet Technology Innovation Center Supported by Virginia’s Center for Innovative Technology Statewide University Partners - Governing Board: F Christopher Newport University –William Winter, William Muir, Virginia Electronic Commerce Technology Center / Southeastern Virginia Network (VECTEC/SEVAnet) F George Mason University –Scott Martin, Internet Multimedia Center (ICM) –Steven Ruth, International Center for Applied Studies in IT (ICASIT) F University of Virginia –Alf Weaver, Internet Commerce Group (InterCom) –Jim French, Internet Digital Library F Virginia Tech –Edward Fox, Digital Library Research Laboratory (DLRL), CC, CS –Scott Midkiff, Center for Wireless Telecomm. (CWT), VTISC, ECpE

4. Digital Library Courseware F http://ei.cs.vt.edu/~dlib/ F WWW pages or large PDF copy files F CourseInfo quizzes based on books by Michael Lesk (MKP.com) and William Arms (MIT Press) F Contents based on books, with other popular topics added (e.g., agents) F Separate pages to supplement: Definitions, Resources (People, Projects), and References

5. JCDL 2001  First Joint ACM/IEEE Conference on Digital Libraries (+ NSF DLI-2 PI mtg) F http://www.jcdl.org F June 24-28, 2001 in Roanoke, VA F Conference Committee: F General Chair: Edward A. Fox, Virginia Tech F Program Chair: Christine Borgman, UCLA F Treasurer: Neil Rowe, Naval Postgraduate School F Posters Chair: Craig Nevill-Manning, Rutgers U.

6. Why this topic today? F Many users (patrons) prefer digital libraries to traditional libraries or the Web F Digital library collections often are free or less expensive, so are heavily used F Most publishers are working toward digital libraries to allow access to their content F Computing as well as library and information science professionals are key players in building digital libraries

7. Outline F Grand Challenge – WHY ! F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

8. Libraries of the Future JCR Licklider, 1965, MIT Press World Nation State City Community

9. Licklider – Unified Theory? F Not ready in 1960s F Analog – unified field theory in physics F “Mess” today – segmented field, specialities –Database Knowledge Content Mgmnt –Multimedia, Hypermedia, Hypertext –Logic, Algebra, Artificial Intelligence, … F Expensive, annoying for users –Don’t know where to look –Don’t know how to use services

11. Computing (flops) Digital content Communicat i ons (bandwidth, connectivity) Locating Digital Libraries in Computing and Communications Technology Space Digital Libraries technology trajectory: intellectual access to globally distributed information lessmore (Slide from S. Griffin, NSF)

12. Grand Challenges Can F Mobilize the community F Spur creativity F Lead to important benefits in society F Push researchers to develop relevant theories F Force people to work in teams/groups F Convince funding agencies to invest F Help bring about integration of systems, interoperability, and seamless interfaces

13. DL Challenges F World Digital Library (Libraries) F Preservation - so people with trust DLs F Scalability, sustainability, interoperability F (Supporting infrastructure - networks, …) F DL industry - critical mass by covering libraries, archives, museums, corporate info, govt info, personal info - “quality WWW” integrating IR, HT, MM,... –Need tools & methods to make them easier to build

14. DLs: Why of Global Interest? F National projects can preserve antiquities and heritage: cultural, historical, linguistic, scholarly F Knowledge and information are essential to economic and technological growth, education F DL - a domain for international collaboration –wherein all can contribute and benefit –which leverages investment in networking –which provides useful content on Internet & WWW –which will tie nations and peoples together more strongly and through deeper understanding

15. Borgman et al.: Workshop Report on Social Aspects of Digital Libraries: http://www-lis.gseis. ucla.edu/DL/ Information Life Cycle

16. Digital Libraries --- Objectives F World Lit.: 24hr / 7day / from desktop F Integrated “super” information systems: 5S: streams, structures, spaces, scenarios, societies F Ubiquitous, Higher Quality, Lower Cost F Education, Knowledge Sharing, Discovery F Disintermediation -> Collaboration F Universities Reclaim Property F Interactive Courseware, Student Works F Scalable, Sustainable, Usable, Useful

17. DL-Related Timeline 1985 CoRR DLI ETDs 1990 xxx 1995 XML OAI 2000 NCSTRL PDF Proposed Ugrad DL NDLTD DLI2 HyperCard TEI SGML PCs Hypertext Conf. DCRDF CSTR NSDL Scholarly EPub in U’s JPEG, MPEG MPEG-7 WWW Java

18. Core of DL F Collecting –Authoring, Repositories, Archives, Museums, … F Organizing –Packaging of Data and Metadata, Storing –Naming/Identifying and Cataloging –Classification, Clustering, … F Serving –Indexing, Linking, Summarizing, Visualizing –Browsing, Accessing, Searching, Filtering, Retrieving, Distributing, Using, …

19. DL Components User Interfaces Workflow Mgr DBMS Search Engines, Classifiers, … Data, MM Info Gateways Repository Rights Mgr MM/ HT Renderer

20. Digital Libraries Shorten the Chain from Editor Publisher A&I Consolidator Library Reviewer

21. DL = Users Direct (Organized Artifact Mediated Communication) Author Reader Digital Library Editor Reviewer Teacher Learner LibrarianDr.Patient

22. Benefits F Ease of use F Effectiveness F “The benefits of digital libraries will not be appreciated unless they are easy to use effectively.” - IITA Workshop report

23. Outline F Grand Challenge F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

25. PetaPlex Top View 4 ft. side

26. PetaPlex Side View 4 ft. wide 8 ft. high Roles: * Support * Cooling * Power 15 shelves

27. PetaPlex Complex FRONT END MACHINE RS/6000, 1G RAM, 4 Proc. Nanoserver Service Machine 1 Service Machine 2 Service Machine 3 Service Machine 4

28. PetaPlex F Digital Library Machine (“super” object store): Parallel computer / storage utility F Research: inverted files, video server, … F Knowledge Systems Incorporated is supplying VT-PetaPlex-1 with 2.5 terabytes through 100 nodes : u Net connection + 25GB disk + 233 MHz Pentium + Linux

29. Structured Video Browser (making video into hypermedia) www.learn.umd.edu F IBrowse F Expository multimedia F Narrative Structures

30. ICU Information and Communication University Users Web Search Engines WWW Servlet Engine Web Server OS DB Search Server Servlet MPEG-7 Description Module 1 2 3 4 5 3’ 4’ 5’ MPEG-7 Image Library Systems Tech. t MPEG-7 Image Library Systems

31. t MPEG-7 Video Library Systems Tech. ICU Information and Communication University MPEG-7 Video Library Systems Tech. Video Data Description Generator Description Schemes Design Tool Description Scheme Meta Database Video Database Retrieval Server Module Player Presentation Module Architecture

32. LMDS offers a LOT of bandwidth (comparison to previous auctions) 020040060080010001200 MHz Interactive & Video Data Wireless Communications Service PCS D-F Block Digital Audio Radio Service Cellular Unserved PCS A-C Block DBS MMDS LMDS LMDS is: - 1300 MHz in two “Blocks” ( 28-31 GHz) - Over 2X bandwidth of AM/FM radio, VHF/UHF television, and Cellular telephone combined. - More than sum of previous 16 auctions

34. SPIRE Visualization

35. CAVE-ETD F CAVE-ETD is a simulation of a library that runs in a CAVE (VR environment). F Populated with a subset of ETD records. Main Foyer room

36. Reading Book Abstract

37. Integrated CCLINC Translingual Information System DARPA Extraction What is the north korean movement in the front line? CCLINC SERVER Info Detection Summarization It seems that North Korea launch a missile again After North Korea launched a Daipodong missile last month, NK is perceived to proceed to an additional test launch. Korea, US and Japan enter into an alert state, and prepare for a joint response policy. Korea estimates that the additional launch will be on 09/05. Japan estimates that NK’s missile range is short. US information says that there is no sign of launch yet. Translation What is the status of nk missile launch against japan? BugHanI IlBonE Ddo MiSaIlEul BalSaHan Deus HaDa 2-way Speech Transation

38. Outline F Grand Challenge F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

39. Definitions F Library ++ (library+archive+museum+…) F Distributed information system + organization + effective interface F User community + collection + services F Digital objects, repositories, IPR management, handles, indexes, federated search, hyperbase, annotation

40. Definition: Digital Libraries are complex systems that F help satisfy info needs of users (societies) F provide info services (scenarios) F organize info in usable ways (structures) F present info in usable ways (spaces) F communicate info with users (streams)

41. 5S Layers Societies Scenarios Spaces Structures Streams

42. Definition: 5S Framework F Societies: interacting people (, computers) F Scenarios: services, functions, operations, methods F Spaces: domains + constraints (e.g., distance, adjacency): 2D, vector, probability F Structures: relations, trees, nodes and arcs F Streams: sequences of items (text, audio, video, network traffic) F (5 Element System: Fire, Wood, Earth, Metal, Water)

43. 5S: Combinations F Societies + Scenarios = user model F Societies + Scenarios + Spaces = user interface F Streams + Structures = markup F Streams + Structures + Scenarios = object F Structures + Scenarios = DBMS

44. Outline F Grand Challenge F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

45. Complex to Simple MARC ($50)Dublin Core (DC)

46. Author‘s tools www.physik.uni-oldenburg.de/EPS/mmm

48. DL Components User Interfaces Workflow Mgr DBMS Search Engines, Classifiers, … Data, MM Info Gateways Repository Rights Mgr MM/ HT Renderer

49. Open Archives Initiative OAI www.openarchives.org openarchives@openarchives.org

50. Original Open Archives Members F American Physical Society F California Digital Library F Caltech F Coalition for Networked Info. F Cornell University F Harvard University F Library of Congress F Los Alamos Nat’l Lab F Mellon Foundation F NASA Langley Research Cntr F Old Dominion University F Stanford University F U. of Ghent F U. of Surrey F U. of Southampton F Vanderbilt University F Virginia Tech F Washington University

51. Approaches to Open Archives Build By Discipline Build By Institution

52. Approaches to Open Archives Build By Discipline Build By Institution Author Category Interdisciplinary Year Language Query …

53. OAi Philosophy F Self-archiving = submission mechanism F Long-term storage system = archive F Open interface = harvesting mechanism F Data provider + service provider F Start with “gray literature” –e-prints/pre-prints, reports, dissertations, …

54. Archive of Digital Objects Archive Access Protocol Handle (ID) Digital object terms and conditions

55. OAI – Repository Perspective Required: Protocol DO MDO

56. OAI – Black Box Perspective OA 1OA 2OA 4OA 3OA 5OA 6OA 7

57. Black Box OAI-ETD Perspective ISTEC (Ibero America) PhysDisNSYSU (Taiwan) ADT (Australia) BN.PT (Portugal) www.theses.orgCyberTheses (Francophone) VTDissert.Online (Germany) MITOhioLINKCBUC (Catalunya) NDC (Greece) SEALS (S.Africa) CICU. Bergen (Norway) … …

58. CS Teaching Center (CSTC) F Collection of reviewed online resources used to aid in teaching of Computer Science F Supports author submission and peer-review process for new ACM Journal of Educational Resources In Computing (JERIC) F Connected with NSDL (NSF 00-44) F http://www.cstc.org

59. W3C Web Characterization Repository F Online database of metadata related to publications, tools and data sets dealing with Web characterization F Project of the Web Characterization Activity working group of the World-Wide-Web Consortium (www.w3c.org/WCA) F http://purl.org/net/repository

60. OAI Repository Explorer F Serves as a compliancy test F Allows browsing of open archives using only OAI protocol F Sends requests on behalf of user, parses and checks responses and displays browsable interface F Will detect most discrepancies in protocol F http://purl.org/net/explorer

61. Tiered Model of Interoperability Mediator services Metadata harvesting Document models

63. Outline F Grand Challenge F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

64. (6 slides from Lee Zia, NSF) Presidential Directive - 12/17/1999 Subject: Use of Information Technology to Improve Our Society “ 13. The Secretary of the Smithsonian Institution, the Director of the National Science Foundation, the Director of the National Park Service, and the Director of the Institute of Museum and Library Services shall work with the private sector and cultural and educational institutions across the country to create a Digital Library of Education to house this country's cultural and educational resources.”

65. Programmatic History Digital Libraries Initiative (DLI 1) - NSF/NASA/ARPA, FY 94-97 DLI 2 - NSF, et al., initiated in FY 98, continuing Education FY 98-99 DLI 2 Special Emphasis in Undergrad NSDL Program NSF: FY 00-02 DL Operational Fall, 2002 DLs & UG Earth Systems Education initiated FY 99, continuing

66. Vision A Learning Environments and Resources Network for SMET Education (LEARNS) F Designed to meet the needs of learners, in both individual and collaborative settings F Constructed to enable dynamic use of a broad array of materials for learning, primarily in digital format F Managed actively to promote reliable anytime - anywhere access to quality collections and services, available both within and without the network (from www.nsf.gov/nsdl)

67. “The network is the library.”

68. LEARNS Connects: Users: students, educators, life-long learners Content: structured learning materials; large real- time or archived datasets; audio, images, animations; primary sources; digital learning objects (e.g. applets); interactive (virtual, remote) laboratories;... Tools: search; refer; validate; integrate; create; customize; publish; share; notify; collaborate;...

69. Expectations of Tracks F Core Integration: to coordinate a distributed alliance of resource collection and service providers, and to ensure reliable and extensible access to and usability of the resulting network of learning environments and resources F Collections: to aggregate and actively manage a subset of the digital library’s content within a coherent theme or specialty F Services: to increase the impact, reach, efficiency, and value of the digital library in its fully operational form F Targeted Research: to have immediate impact on one or more of the other three tracks

70. Selected DL2 Ugrad Projects/Topics UNCW, Eduprise, TCNJ, VT,…: iLumina Project IMS, CS, Math, Viz., … Columbia UniversityEarth sciences Stanford UniversityMedicine (images) U. California BerkeleyEngineering University of MarylandK-12 education U. Texas at AustinPhysical anthropology

71. Tracks & 29 Projects F 6 Core Integration: Columbia, Cornell, E.Michigan/MERIT, UCAR, UCB, U- Missouri/NCSA (Biology, Eng., Teacher Ed.) F 13 Collections: Atmosphere, Biology, Biosciences, Earth Systems, Engineering, Health Sciences, Math F 9 Services: Competitive Intelligence, Component Environment, Earth Systems J., Metadata NLP, Managing LOs, Peer Review, Video F 1 Targeted Research: Paths

72. NSDL Spine full-service collections full-service collections NSDL Collections referenced items & collections referenced items & collections Referenced Items & Collections NSDL Services NSDL Services Other NSDL Services CI Services discussion CI Services personalization CI Services topic-map registry CI Services query transform Core Collection- Usage Services annotation Core Collection- Building Services protocol mediation Core Collection- Building Services persistence Core Collection- Building Services harvesting Portals & Clients Portals & Clients Portals & Clients (Slide from Dave Fulker, Bill Arms – 11/2/2000)

74. CS Teaching Center (CSTC) F Instead of building large, expensive multimedia packages, that become obsolete and are difficult to re-use, concentrate on small knowledge units. F Learners benefit from having well-crafted modules that have been reviewed and tested. F Use digital libraries to build a powerful base of support for learners, upon which a variety of courses, self-study tutorials & reference resources can be built. F ACM Education Board and SIG support, new NSF grant with COLLEGIS Research Institute and others …

76. Browsing (1)

77. Browsing (2)

81. A Digital Library Case Study F Domain: graduate education, research F Genre: ETDs = electronic theses & dissertations F Submission: http://etd.vt.edu F Collection: http://www.theses.org Project: Networked Digital Library of Theses & Dissertations http://www.ndltd.org (NDLTD – remember: ND LTD / NDL TD) (also, newer NUDL: Networked University Digital Library, with e-courseware, etc.)

82. Grad Program Library IT Ed. (Tech)

83. The Networked Digital Library of Theses and Dissertations www.NDLTD.org Leader of the Worldwide ETD (Electronic Thesis and Dissertation) Initiative Training Authors Expanding Access Preserving Knowledge Improving Graduate Education Enhancing Scholarly Communication Empowering Students & Universities

84. What are the long term goals? F Attract all TDs/yr: 50K D-US, 25K D-Germany, 10K TD-Canada, … F >200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …) F Dramatic increase in knowledge sharing: literature reviews, bibliographies, … F Services providing lifelong access for students: browse, search, prior searches, citation links F Hundreds/thousands of downloads / year / work

85. Student Defends & Finalizes ETD My Thesis ETD

86. Student Gets Committee Signatures and Submits ETD Signed Grad School

87. Graduate School Approves ETD, Student is Graduated Ph.D.

88. Library Catalogs ETD, Access is Opened to the New Research WWW NDLTD

89. User Search Support (multilingual, XML) Note: All groups shown are connected with NDLTD.

90. Access Possibilities Web search engines library catalog clients www. theses. org www. openarchives. org 3 rd Party Services (e.g., UMI) Virginia Tech National Library of Portugal CBUC (Spain) Ohio Link MITNational Projects: AU, GE, …

91. Status of the Local Project F Approved by university governance Spring 1996; required starting 1/1/97 F Submission & access software in place F Submission workshops for students (and faculty) occur often: beginner/adv. F Faculty training as part of Faculty Development Initiative F Over 3000 ETDs in collection – some have audio, video, large images, software, …

92. US University Members (44) F Penn. State University F Rochester Institute of Tech. F U. of Colorado Health Science Center F U. of Florida F U. of Georgia F University of Hawaii, Manoa F U. of Iowa F U. of Kentucky F U. of Maine F U. of North Texas – required since 8/99 F U. of Oklahoma F U. of South Florida F U. of Tennessee, Knoxville F U. of Tennessee, Memphis F U. of Texas at Austin – required in 2001 F U. of Virginia F U. Wisconsin - Madison F Vanderbilt U. F Virginia Commonwealth U. F Virginia Tech - required since 1/97 F West Virginia U. - required fall 1998 F Western Michigan U. F Worcester Polytechnic Inst. F Air University (Alabama) F Baylor University F Brigham Young University (part, whole) F Caltech F Clemson University F College of William & Mary F Concordia University (Illinois) F East Carolina University F East Tenn. State U. – require fall 2000 F Florida Institute of Technology F Florida International University F George Washington University F Louisiana State University F Marshall University (W. Va.) F Miami University of Ohio F Michigan Tech F Mississippi State University F MIT F Naval Postgraduate School (CA) F New Mexico Tech F North Carolina State University

93. OhioLINK F Statewide Consortium F Represents 79 colleges, universities, libraries F Public Universities F Private Universities and Colleges F 2-Year Colleges F Only a few (e.g., Miami U. of Ohio) are also NDLTD members on their own

94. National / Regional Projects F Australia –U. New South Wales (lead) –U. of Melbourne –U. of Queensland –U. of Sydney –Australian National U. –Curtin U. of Technology –Griffith U. F Germany –Humboldt University (lead) –3 other universities –5 learned societies: Math, Physics, Chemistry, Sociology, Education –1 computing center –2 major libraries F Consorci de Biblioteques Universitàries de Catalunya, as group, www.cbuc.es: –Universitat de Barcelona –Universitat Autonòma de Barcelona –Universitat Politècnica de Catalunya –Universitat Pompeu Fabra –Universitat de Girona –Universitat de Lleida –Universitat Rovira i Virgili –Universitat Oberta de Catalunya –Biblioteca de Catalunya F South Africa: ECHEA/SEALS F India, Portugal, …

95. Other Countries with Members F Belgium F Brazil F Canada F Germany F Hong Kong F India F Italy F Korea F Mexico F Netherland F Norway F Russia F Singapore F S. Africa F S. Korea F Spain F Taiwan F UK

96. ETD Initiative (and UMI) Students Learn about DL, EPub TDs become more expressive N. Amer. (T)Ds are accessible, archived Global TDs become more accessible, archived UMI Universities

97. Convene Local Planning Group ETD

98. Build Local ETD Site Digital Library Policies Inspection/Approval Workshop/Training ETD

99. Responsibilities F Handle local education and collection –Contact information for helpers –Archive F Utilize standards –Metadata: MARC / DC-based concensus specification F Share metadata –Union services, mirrored services F Allow access –www.theses.org / www.dissertations.org –Open Archives Initiative (www.openarchives.org)

102. MARIAN Layers Database Layer Search Engine Layer User Information Layer User Interface Layer User

111. Remember F Grand Challenge F Scaling / Technology F Framework, Theory F Simplification: DC, OAI F Example Applications

112. Conclusions F Consider DLs: to use, to teach, to add to, to build F Education is one important application of DLs F Cultural heritage, linguistic diversity, new knowledge – all are important to preserve F Technology opens up exciting opportunities in DLs to yield seamless “super” information systems F Having a framework and theory may lead to better (more effective) systems and broader applicability F Interoperability is part of the DL grand challenge

113. URLs  http://fox.cs.vt.edu  http://ei.cs.vt.edu/~dlib (Courseware)  http://www.dlib.org (D-Lib Magazine)  www.smete.org and later www.nsf.gov/nsdl  www.ndltd.org and www.theses.org  www.cstc.org (CSTC and JERIC)  www.openarchives.org  www.jcdl.org (JCDL’2001 – June 24-28)