Digital Libraries, Universities, and Scholarly Communication EUNIS 2000 Poznań, April, 2000 Edward A. Fox CC CS DLRL Internet TIC Virginia Tech, Blacksburg, VA, USA

Acknowledgements (Selected) F Sponsors: ACM, Adobe, ARL, Belgian Science Found., CLIR, DARPA, IBM, LANL, Microsoft, NSF, OCLC, SPARC, US Dept. of Ed. (FIPSE), … F VT Faculty/Staff: Tony Atkins, Thomas Dunbar, John Eaton, Gwen Ewing, Peter Haggerty, Gary Hooper, Gail McMillan, Len Peters, James Powell  VT Students: Emilio Arce, Fernando Das Neves, Brian DeVane, Robert France, Marcos Goncalves, Scott Guyer, Robert Hall, Neill Kipp, Paul Mather, Tim McGonigle, Todd Miller, Constantinos Phanouriou, William Schweiker, Ohm Sornil, Hussein Suleman, Patrick Van Metre, Laura Weiss

Virginia Tech Background F Largest university in Virginia, land-grant, football, town population 35K plus 25K students F Blacksburg Electronic Village, since 1992, with > 80% of community on Internet F Net.Work.Virginia, largest ATM network, with over 750 sites, for education, research, gov’t F LMDS, Local Multipoint Distribution Service, gigabit wireless networking - 1/3 of Virginia F Math Emporium, 500 workstations F Faculty Development Initiative, round 2

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)

Digital Libraries SGML (1985) PDF (1992) NSF DLI (1994) Library Cancellations (1988) University Scholarly Electronic Pub. (1988) Info. Literacy (1995) Improving Education Internet (1984) WWW (1994) Multimedia (1986)

Computing (flops) Digital content Communicat i ons (bandwidth, connectivity) Locating Digital Libraries in Computing and Communications Technology Space Digital Libraries technology trajectory: intellectual access to globally distributed information lessmore

Digital Libraries Shorten the Chain from Editor Publisher A&I Consolidator Library Reviewer

DLs Shorten the Chain to Author Reader Digital Library Editor Reviewer Teacher Learner Librarian

Digital Libraries --- Objectives F World Lit.: 24hr / 7day / from desktop F Integrated “super” information systems: 5S: streams, structures, spaces, scenarios, societies F Ubiquitous, Higher Quality, Lower Cost F Education, Knowledge Sharing, Discovery F Disintermediation -> Collaboration F Universities Reclaim Property F Interactive Courseware, Student Works F Scalable, Sustainable, Usable, Useful

How do universities and digital libraries relate? F Each U. will have its own digital library. Hence there will be large numbers (i.e., critical mass). F All students will learn how to use and how to “feed” digital libraries (and bring those habits to future work as needs and skills). F Work toward NUDL : –Networked University Digital Library

DLs: Why of Global Interest? F National projects can preserve antiquities and heritage: cultural, historical, linguistic, scholarly F Knowledge and information are essential to economic and technological growth, education F DL - a domain for international collaboration –wherein all can contribute and benefit –which leverages investment in networking –which provides useful content on Internet & WWW –which will tie nations and peoples together more strongly and through deeper understanding

Definition: Digital Libraries are complex systems that F help satisfy info needs of users (societies) F provide info services (scenarios) F organize info in usable ways (structures) F present info in usable ways (spaces) F communicate info with users (streams)

5S Layers Societies Scenarios Spaces Structures Streams

DL Challenges F Preservation - so people with trust DLs F Supporting infrastructure - networks,... F Scalability, sustainability, interoperability F DL industry - critical mass by covering libraries, archives, museums, corporate info, govt info, personal info - “quality WWW” integrating IR, HT, MM,... –Need tools & methods to make them easier to build

Enhancing Learning with DLs

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)

Open Archives initiative OAi

OAi Philosophy F Self-archiving = submission mechanism F Long-term storage system = archive F Open interface = harvesting mechanism F Data provider + service provider F Start with “gray literature” –e-prints/pre-prints, reports, dissertations, …

Tiered Model of Interoperability Mediator services Metadata harvesting Document models

Repository of Digital Objects Repository Access Protocol handle Digital object terms and conditions

Open Archives (protoproto) F ArXiv & Los Alamos National Lab F CogPrints & U. Southampton F NACA & NASA (reports) F NCSTRL & Cornell U. F NDLTD & Virginia Tech F RePEc & U. Surrey F Total of around 200K records

Original Open Archives Members F Caroline Arms, Library of Congress F Leslie Carr, University of Southampton F Mark Doyle, American Physical Society F Dale Flecker, Harvard University F Edward A. Fox, Virginia Tech F Michael Friedman, HighWire Press, Stanford U. F Paul M. Gherman, Vanderbilt U. & SPARC F Paul Ginsparg, Los Alamos National Lab. & xxx F Stevan Harnad, University of Southampton F Thomas Krichel, University of Surrey & RePEc F Carl Lagoze, Cornell University …

Original Open Archives Members cont’d F Rick Luce, Los Alamos National Laboratory F Clifford Lynch, Coalition for Networked Info. F Kurt Maly, Old Dominion University F Michael Nelson, NASA Langley Research Center F John Ober, California Digital Library F Bob Parks, Washington University & EconWPA F Herbert Van de Sompel, University of Ghent F Eric F. Van de Velde, Caltech F Don Waters, The Andrew W. Mellon Foundation F Ken Weiss, California Digital Library

Open Archives Future F EconWPA (U. Washington) F e-biomed -> PubMed Central (NIH) F PubScience (DOE) F Clinical Medicine Netprints (+ other HighWire Press holdings ) F University ePub (California Digital Library) F All public e-prints (MIT) F Scholar’s Forum (Caltech) F Int’l: CERN, Germany, India, Mexico, … F Goal: millions of books/articles/reports / yr

Approaches to Open Archives Build By Discipline Build By Institution

Approaches to Open Archives Build By Discipline Build By Institution Author Category Interdisciplinary Year Language Query …

Open Archives initiative History F xxx at LANL = Los Alamos National Laboratory (Ginsparg) for high-energy physics F CSTR + WATERS = NCSTRL (Lagoze) F xxx + NCSTRL = CoRR collaboration F UPS (Universal Preprint Service) – 1999 mtg –Santa Fe, Oct , protoproto –LANL, CNI, DLF, Mellon, … –Dublin Core (DC), XML –Herbert Van de Sompel (U. Ghent, SFX) … –Dienst protocol and software (Lagoze)

Open Archives initiative (OAi) F Santa Fe Convention (see Feb. D-Lib Magazine article) F Renamed late 1999 as OAi F Next mtg June 3, San Antonio, between HT’00 & DL’00 F Then Sept. 21 as workshop of ECDL ’2000, Lisbon F Archives -> Open Archives –Support unique archive identifiers –Implement Open Archives Metadata Set (DC-based, using XML) –Implement Dienst harvesting interface –Register the archive F Build tools, layer other services: linking, searching, …

Mechanisms F Sharing –Join federation, run software –Make metadata and archive available F Aggregating –By discipline –By institution –By genre F Automating –Workflow –Harvesting and providing services –Federated searching –Dynamic linking (e.g., with SFX)

Virginia Tech Projects F OAi Repository Explorer F MARC XML-DTD F W3C Web Characterization Repository F Computer Science Teaching Centre (CSTC) F Networked Digital Library of Theses and Dissertations (NDLTD)

OAi Repository Explorer F Serves as a compliancy test F Allows browsing of open archives using only OAi protocol F Sends requests on behalf of user, parses and checks responses and displays browsable interface F Will detect most discrepancies in protocol F

MARC XML-DTD F XML Transport format for US-MARC records F Standardized metadata exchange format for traditional library services joining OAi

W3C Web Characterization Repository F Online database of metadata related to publications, tools and data sets dealing with Web characterization F Project of the Web Characterization Activity working group of the World-Wide-Web Consortium ( F

CS Teaching Center (CSTC) F Collection of reviewed online resources used to aid in teaching of Computer Science F Supports author submission and peer-review process for new ACM Journal of Educational Resources In Computing (JERIC) F Connected with NSDL (NSF 00-44) F

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)

The Networked Digital Library of Theses and Dissertations Leader of the Worldwide ETD (Electronic Thesis and Dissertation) Initiative Training Authors Expanding Access Preserving Knowledge Improving Graduate Education Enhancing Scholarly Communication Empowering Students & Universities

A Digital Library Case Study F Domain: graduate education, research F Genre:ETDs=electronic theses & dissertations F Submission: F Collection: Project: Networked Digital Library of Theses & Dissertations (NDLTD)

Grad Program Library IT Ed. (Tech)

Media ETD Web Site ETDs Got Your Interest? Graduate Students Singapore AM Chronicle of Higher Ed. National Public Radio NY Times... U. Laval

Key Ideas: Networked infrastructure Scalability Education is the rationale University collaboration Workflow, automation Authors must submit Maximal Access PDF, SGML, MM, MARC, DC, URNs, Federated search Standards 8th graders vs. grads

What led to today’s meeting? F 1987 mtg in Ann Arbor: UMI, VT, … F 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities with 3 reps each F 1993 mtg in Atlanta to start Monticello Electronic Library (MEL): SURA, SOLINET F 1994 mtg in Blacksburg re ETD project: std of PDF + SGML + multimedia objects F 1996 funding by SURA, US Dept. of Education (FIPSE) for regional, national projects F 1997 meetings in UK, Germany,... F 1998 – 1 st symposium – Memphis (20) F 1999 – 2 nd symposium – Blacksburg (70) F 2000 – 3 rd symposium – St. Petersburg (225) -> Caltech

What are the long term goals? F 400K US students / year getting grad degrees are exposed / involved F 200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …) F Dramatic increase in knowledge sharing: literature reviews, bibliographies, … F Services providing lifelong access for students: browse, search, prior searches, citation links F Hundreds/thousands of downloads / year / work

F Aiding universities to enhance graduate education, publishing and IPR efforts F Helping improve the availability and content of theses and dissertations F Educating ALL future scholars so they can publish electronically and effectively use digital libraries (i.e., are Information Literate and can be more expressive) What are we doing?

NDLTD Computer Resources Research Literature Student Prepares Thesis/Dissertation

Student Defends & Finalizes ETD My Thesis ETD

Student Gets Committee Signatures and Submits ETD Signed Grad School

Graduate School Approves ETD, Student is Graduated Ph.D.

Library Catalogs ETD, Access is Opened to the New Research WWW NDLTD

Status of the Local Project F Approved by university governance Spring 1996; required starting 1/1/97 F Submission & access software in place F Submission workshops for students (and faculty) occur often: beginner/adv. F Faculty training as part of Faculty Development Initiative F Over 2400 ETDs in collection – some have audio, video, large images, software, …

Institutional Members F Coalition for Networked Information (CNI) F Committee on Institutional Cooperation (CIC) F Diplomica.com F Dissertation.com F Dissertationen Online (Germany) F Ibero-American Science & Technology Education Consortium (ISTEC, F National Library of Portugal (for all universities) F Organization of American States (SEDI/OAS) F UNESCO (

US University Members (41) F U. of Florida F U. of Georgia F University of Hawaii, Manoa F U. of Iowa F U. of Kentucky F U. of Maine F U. of North Texas – required since 8/99 F U. of Oklahoma F U. of South Florida F U. of Tennessee, Knoxville F U. of Tennessee, Memphis F U. of Texas at Austin F U. of Virginia F U. Wisconsin - Madison F Vanderbilt U. F Virginia Commonwealth U. F Virginia Tech - required since 1/97 F West Virginia U. - required fall 1998 F Western Michigan U. F Worcester Polytechnic Inst. F Air University (Alabama) F Baylor University F Brigham Young University F Caltech F Clemson University F College of William & Mary F Concordia University (Illinois) F East Carolina University F East Tenn. State U. – require fall 2000 F Florida Institute of Tech. F Florida International University F George Washington University F Marshall University (W. Va.) F Miami U. of Ohio F MIT F Michigan Tech F Naval Postgraduate School (CA) F North Carolina State U. F Penn. State University F Rochester Institute of Tech. F U. of Colorado Health Science Center

Australian Project Members F U. New South Wales (lead institution) F U. of Melbourne F U. of Queensland F U. of Sydney F Australian National University F Curtin U. of Technology F Griffith U.

German Project Members F Humboldt University (lead institution) F 3 other universities F 5 learned societies –Mathematics, Physics, Chemistry, Sociology, Education F 1 computing center F 2 major libraries

CBUC ( Spain) F Consorci de Biblioteques Universitàries de Catalunya, as group, with 9 members: –Universitat de Barcelona –Universitat Autonòma de Barcelona –Universitat Politècnica de Catalunya –Universitat Pompeu Fabra –Universitat de Girona –Universitat de Lleida –Universitat Rovira i Virgili –Universitat Oberta de Catalunya –Biblioteca de Catalunya

Other International Members F Chinese University of Hong Kong F Chungnam National U. (S. Korea - CS) F City University, London (UK) F Darmstadt U. of Tech. (Germany) F Free University of Berlin (GE - Vet. Med.) F Gyeongsang National U. (Korea) F India Institute of Tech., Bombay (India) F Nanyang Technological U. (Singapore, pt) F National U. of Singapore (Singapore, pt)

Other International Members cont’d F Polytechnic University of Valencia (Spain) F Rhodes U. (South Africa) F St. Petersburg St. Tech.U (Russia) F Univ. de las Américas Puebla (Mexico) F Univ. of Alicante (Spain) F Univ. of Pisa (Italy) F U. Laval; U. of Guelph; U. Waterloo; F Wilfrid Laurier U. (Canada), …

For professional societies F Like “writing across the curriculum”, e.g., Chemical Markup Language, MathML, … F Besides writing: computing/communications, information literacy, personal digital library management, tool use, research methods, collaboration, archiving/preservation F Data sets, communities of users of them F Classification systems / browsing / searching F NRC’s “On becoming a researcher”

NUDL F 1/15/99 NUDL proposal to NSF under DLI2 international program –VT: Library, Grad School, Industrial&Systems Eng. –Partners: UK (2), Singapore, Russia, Korea, Greece, Germany, plus Iberoamerican group (Spain, Portugal, Argentina, Brazil, Chile, Mexico) –Problems: Multilingual search, multimedia submissions, requirements/usability, … F Start with ETDs, then expand to other student works, portfolios, data sets, (CS) courseware,...

Who are sponsors / cooperators? F Funding, Donations of hardware/software –SURA –US Dept. of Education (FIPSE) –Adobe Systems –IBM –Microsoft –OCLC F Others Serving on Steering Committee –National/Regional Projects: Australia, French speaking group, Germany, IberoAmerica (ISTEC), UK (UTOG) –CGS, National Lib. Canada, NSF, OAS, SOLINET, UMI, UNESCO,...

Relationship with publishers F Concern of faculty and students that still wish to publish books or journal articles, voiced: campus, Chronicle, NPR, Times F Solution: Approval Form gives students, faculty choices on access, when to change access condition; use IPR controls in DL F Solution: by case, work with publishers and publisher associations to increase access – AAP, AAUP – AAAS, ACM, ACS, Elsevier,...

Some responses from publishers F ACM: need to acknowledge copyright F Elsevier: need to acknowledge copyright F IEEE-CS: endorse initiative F ACS: After first publication, can release F Textbook publishers: different market, manuscript significantly reworked F General: restricting access to local campus will not cause any problems

How does this relate to UMI? F Generally, they are independent decisions. F 1987 UMI workshop was first to explore ETDs. F UMI wrote support letter for US Dept. of Ed. proposal. F UMI is on Steering Committee. F ProQuest Direct pilot of scanning works started 1/1/97, with free 2 yr access to front part. F We are collaborating on: – accepting electronic author submissions – standards (e.g., representation)

ETD Initiative (and UMI) Students Learn about DL, EPub TDs become more expressive N. Amer. (T)Ds are accessible, archived Global TDs become more accessible, archived UMI Universities

User Search Support (multilingual, XML) Note: All groups shown are connected with NDLTD.

F James Powell student project, D-Lib Magazine description in Sept F XML description of each site –type of search engine / service –language –coverage (for resource discovery) F Adding Z39.50 gateway capability and integrating with MARIAN, along with Harvest and Open Archives protocols

Access Possibilities Web search engines library catalog clients www. theses. org www. openarchives. org 3 rd Party Services (e.g., UMI) Virginia Tech National Library of Portugal CBUC (Spain) Ohio Link MITNational Projects: AU, GE, …

MARIAN F Multiple Access Retrieval of Information with Annotations F (Marian the Librarian …) F Evolved from CODER system to a distributed Online Public Access Catalog (OPAC), then DL backend, now becoming a full DL system F From C/C++ to Java F Future: NDLTD, NUDL, PetaPlex F Use for campus collection management F Use for as centralized system with gateway services

MARIAN Layers Database Layer Search Engine Layer User Information Layer User Interface Layer User

Why might a university want to be involved? F To improve graduate education / better prepare your students / increase their knowledge and visibility F To unlock university information F To save money for students and for the university / improve workflow F To build an important digital library

How can a university get involved? F Select planning/implementation team – Graduate School – Library – Computing / Information Technology – Institutional Research / Educ. Tech. F Send us letter, give us contact names – F Adapt Virginia Tech solution – Build interest and consensus – Start trial / allow optional submission

Contact Our Project Team Phone Call Visit Video Tape

Convene Local Planning Group ETD

Build Local ETD Site Digital Library Policies Inspection/Approval Workshop/Training ETD

Support Services Developed F WWW site with > 300 Mb, CD, videotape F Automated submission system (MySQL, UNIX, WWW scripts - grad school/library) F Student guidelines, style sheets, multimedia training materials, FAQs, press info F SGML and XML DTDs for ETDs F SGML to HTML (web generator) F LaTeX, Word templates, converters F FTP site for PS to PDF conversion with UNIX distiller

DL Submission Software F Similar software developed for W3C’s WCA, CSTC, and NDLTD F CSTC version field-tested to manage papers for ACM Digital Libraries ‘99 F May generalize for –conferences –electronic journal –resource description (e.g., courses, Web content)

Support Offered F Documentation, technical support F , listservs F Donations: Adobe, Microsoft F Evaluation: instruments, analysis - solutions/statistics F (Temporary storage / archiving; aid - in setting up an int’l service & archive)

PetaPlex F Digital Library Machine (“super” object store) F Parallel computer / storage utility F Knowledge Systems Incorporated is supplying VT-PetaPlex-1 with –high speed backbone connection –2.5 terabytes through 100 nodes : u Net connection + 25GB disk MHz Pentium + Linux

PetaPlex Complex FRONT END MACHINE RS/6000, 1G RAM, 4 Proc. Nanoserver Service Machine 1 Service Machine 2 Service Machine 3 Service Machine 4

Accessibility Activities / Plans F Interface design (simple, 3D, VR) F Usability studies F Generic multi-lingual support F Support for those with disabilities F Hybrid collection (paper, MARC, abstracts, full-text, multimedia) F Disciplinary classifications, tools F Visualization of results, collection

CAVE Experiments F Use a familiar metaphor –building / floor / room / shelf / book F Rearrange orderings / shelving –use categories, clustering, ranking –use visualization: colors and gaps –study space mappings: physical, logical F Simplify movement for key tasks

SPIRE Visualization

Extending Services - 1 of 2 F Working with publishers –Motivate students: awards, … –Publicize support of NDLTD u ACM, ACS, IEEE-CS, Elsevier, … –Allow students to increase level of access F Arranging preservation –Mirroring worldwide –Involving long-term trusted parties

Extending Services - 2 of 2 F Adding services currently prototyped –annotation and SDI (routing) capabilities –Dublic Core metadata, crosswalk to MARC –support for XML, *ML, preservation –harvesting, federated search F Adding other services planned –building/using citation DB (CiteSeer, SFX, …) –implementing plagiarism check (like “SCAM”)

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)

NCSTRL F F Networked Computer Science Technical Reference Library F CS Technical Reports F 1994 merger of CSTC + WATERS F 1998 integration with LANL server (CoRR) F Federated search, mirrors, Dienst protocol

Digital Library / CS Courseware F F From NSF-funded CS ei.cs.vt.edu F Online quizzes based on book by Michael Lesk (Morgan Kaufmann Publishers) F Contents based on book, with several other popular topics added (e.g., agents) F Separate pages to supplement: Definitions, Resources (People, Projects), and References

CS Teaching Center (CSTC) F Instead of building large, expensive multimedia packages, that become obsolete and are difficult to re-use, concentrate on small knowledge units. F Learners benefit from having well-crafted modules that have been reviewed and tested. F Use digital libraries to build a powerful base of support for learners, upon which a variety of courses, self-study tutorials & reference resources can be built.

Browsing (1)

Browsing (2)

CS -> CSTC -> CRIM F NSF and ACM Education Committee are funding a 2 year project “A Computer Science Teaching Center” - CSTC - F College of NJ, U. Ill. Springfield, Virginia Tech F Focus initially on labs, visualization, multimedia F Multimedia part is also supported by a 2nd grant to Virginia Tech and The George Washington University: (with curricular guidelines also under development)

Curriculum Resources in Interactive Multimedia (CRIM) F MM field needs properly trained personnel F Support this with resources + curricula F Benefits will go to teachers (who have more to build upon) and students (who will have a richer environment for learning F CSTC, CRIM will lead to ACM Journal of Educational Resources in Computing, JERIC F Together these help us move forward: DL for Interactive MM -> CS -> NSDL

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)

SMETE Library -> NSDL (from to NSF DLI-2) F Context: Global movement toward Digital Libraries (see April 1998 CACM) F NSF effort: Science, Mathematics, Engineering, and Technology Education Digital Library (focussed on undergraduates) –3 workshops, yearly increasing funds / new calls F NSDL will operate as a distributed federation, with separate parts for each key discipline, and should lead to a global effort.

Selected NSDL Projects/Topics COLLEGIS Res. Inst.IMS, CS, Math, Viz., … Columbia UniversityEarth sciences Stanford UniversityMedicine (images) U. California BerkeleyEngineering University of MarylandK-12 education U. Texas at AustinPhysical anthropology

Remember! F Digital Libraries (technology base) F OAi (help establish enormous international cooperative of data and service providers) F NDLTD (improve graduate education – join!) F NCSTRL, CSTC, CRIM, JERIC (computing) F NSDL: National Science Digital Library (US)