Digital Libraries, OAI, ETDs, and NDLTD SEALS ETD Meeting October 25, 2000 Edward A. Fox CS DLRL Internet TIC Virginia Tech, Blacksburg, VA, USA

Acknowledgements (Selected) F Sponsors: Adobe, IBM, Microsoft, NSF, OCLC, SOLINET, SURA, UNESCO, US Dept. of Ed. (FIPSE), … F VT Faculty/Staff: Tony Atkins, Thomas Dunbar, Debra Dudley, John Eaton, Gwen Ewing, Peter Haggerty, Gary Hooper, Gail McMillan, Len Peters, James Powell, …  VT Students: Emilio Arce, Fernando Das Neves, Brian DeVane, Robert France, Marcos Goncalves, Scott Guyer, Robert Hall, Neill Kipp, Paul Mather, Tim McGonigle, Todd Miller, Constantinos Phanouriou, William Schweiker, Ohm Sornil, Hussein Suleman, Patrick Van Metre, Laura Weiss, …

JCDL 2001  First Joint ACM/IEEE Conference on Digital Libraries F F June 24-28, 2001 in Roanoke, VA F Conference Committee: F General Chair: Edward A. Fox, Virginia Tech F Program Chair: Christine Borgman, UCLA F Treasurer: Neil Rowe, Naval Postgraduate School F Posters Chair: Craig Nevill-Manning, Rutgers U.

Digital Libraries Shorten the Chain from Editor Publisher A&I Consolidator Library Reviewer

DLs Shorten the Chain to Author Reader Digital Library Editor Reviewer Teacher Learner Librarian

DL Components User Interface Workflow Mgr DBMS Search Engines File System (MM) Web Gateway Repository Rights Mgr

Open Archives Initiative OAI

Open Archives Initiative (OAI) F high-energy physics (Ginsparg, 1991) F Universal Preprint Service protoproto, Oct , 1999, Santa Fe – led by LANL, CNI, DLF, Mellon --> OAi F Santa Fe Convention (see Feb. D-Lib Magazine article) F Follow-on mtgs: Antonio, (ECDL) F Steering Committee, Technical Committee; DLF, CNI, … F Archives -> Open Archives –Support unique archive identifiers –Implement Open Archives metadata set (DC, using XML) –Implement OA harvesting protocol –Register the archive F Build tools, layer other services: linking, searching, …

OAi Philosophy F Self-archiving = submission mechanism F Long-term storage system = archive F Open interface = harvesting mechanism F Data provider + service provider F Start with “gray literature” –e-prints/pre-prints, reports, dissertations, …

Archive of Digital Objects Archive Access Protocol Handle (ID) Digital object terms and conditions

OAI – Repository Perspective Required: Protocol DO MDO

OAI – Black Box Perspective OA 1OA 2OA 4OA 3OA 5OA 6OA 7

The Networked Digital Library of Theses and Dissertations Leader of the Worldwide ETD (Electronic Thesis and Dissertation) Initiative Training Authors Expanding Access Preserving Knowledge Improving Graduate Education Enhancing Scholarly Communication Empowering Students & Universities

A Digital Library Case Study F Domain: graduate education, research F Genre: ETDs = electronic theses & dissertations F Submission: F Collection: Project: Networked Digital Library of Theses & Dissertations (NDLTD – remember: ND LTD / NDL TD) (also, newer NUDL: Networked University Digital Library, with e-courseware, etc.)

Grad Program Library IT Ed. (Tech)

Key Ideas: Networked infrastructure Scalability Education is the rationale University collaboration Workflow, automation Authors must submit Maximal Access PDF, SGML, MM, MARC, DC, URNs, Federated search Standards 8th graders vs. grads

What led to today’s meeting? F 1987 mtg in Ann Arbor: UMI, VT, … F 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities with 3 reps each F 1993 mtg in Atlanta to start Monticello Electronic Library (MEL): SURA, SOLINET F 1994 mtg in Blacksburg re ETD project: std of PDF + SGML + multimedia objects F 1996 funding by SURA, US Dept. of Education (FIPSE) for regional, national projects F 1997 meetings in UK, Germany,... F 1998 – 1 st symposium – Memphis (20) F 1999 – 2 nd symposium – Blacksburg (70) F 2000 – 3 rd symposium – St. Petersburg (225) -> Caltech

F Aiding universities to enhance graduate education, publishing and IPR efforts F Helping improve the availability and content of theses and dissertations F Educating ALL future scholars so they can publish electronically and effectively use digital libraries (i.e., are Information Literate and can be more expressive) –=> leverage to reinvent scholarly communication What are we doing?

What are the long term goals? F Attract all TDs/yr: 50K D-US, 25K D-Germany, 10K TD-Canada, … F >200K/yr rich hypermedia ETDs that may turn into electronic portfolios (images, video, audio, …) F Dramatic increase in knowledge sharing: literature reviews, bibliographies, … F Services providing lifelong access for students: browse, search, prior searches, citation links F Hundreds/thousands of downloads / year / work

ETDs: Library Goals F Improve library services –Better turn-around time –Always available F Reduce work –catalog from e-text –eliminate handling: mailing to UMI, bindery prep, check-out, check-in, reshelving, etc. F Save space

NDLTD Computer Resources Research Literature Student Prepares Thesis/Dissertation

Student Defends & Finalizes ETD My Thesis ETD

Student Gets Committee Signatures and Submits ETD Signed Grad School

Graduate School Approves ETD, Student is Graduated Ph.D.

Library Catalogs ETD, Access is Opened to the New Research WWW NDLTD

Status of the Local Project F Approved by university governance Spring 1996; required starting 1/1/97 F Submission & access software in place F Submission workshops for students (and faculty) occur often: beginner/adv. F Faculty training as part of Faculty Development Initiative F Over 3000 ETDs in collection – some have audio, video, large images, software, …

Archiving ETDs F Every 15 minutes back-ups made of not- yet-approved submissions F Hourly back-ups of newly approved ETDs F Weekly back-ups of entire ETD collection F Copies stored on-site and off-site

VT ETD Cataloging F same as current cataloging policies, except: –author-assigned keywords (not LCSH) –generic (not LC) call no. –fields/subfields as required for computer files –full abstracts F time savings –cataloger familiar with computer files –equipment, software for word processing –5 minutes avg. (10-15 minutes for paper TDs)

Library Costs F $12/vol. for paper thesis processing – catalog, bind, security strip, label, shelve vols./yr. = $11,466 F $3.20/vol. ETD processing – vols./yr. = $3040 F $.07/vol. shelving F $.04/vol. circulation

Costs/Savings at VT F Graduate School stopped shipping to the library 3000 copies of paper TDs/year F Library stopped binding, shelving, and circulating 3000 copies of TDs/year F 166 ft of shelf space saved/year by the library F VT used existing equipment in Library (vs. start-up costs for staff, hardware and software from from a zero-base estimate: $65,000 – see

Institutional Members F Coalition for Networked Information (CNI) F Committee on Institutional Cooperation (CIC) F Diplomica.com F Dissertation.com F Dissertationen Online (Germany) F ETDweb, a Division of Answer4.com F Ibero-American Science & Technology Education Consortium (ISTEC) F National Documentation Centre (NDC), Greece F National Library of Portugal (for all universities) F OCLC Online Computer Library Center F Organization of American States (SEDI/OAS) F Southeastern Library Network (SOLINET) F UNESCO (

National / Regional Projects F Australia –U. New South Wales (lead) –U. of Melbourne –U. of Queensland –U. of Sydney –Australian National U. –Curtin U. of Technology –Griffith U. F Germany –Humboldt University (lead) –3 other universities –5 learned societies: Math, Physics, Chemistry, Sociology, Education –1 computing center –2 major libraries F Consorci de Biblioteques Universitàries de Catalunya, as group, –Universitat de Barcelona –Universitat Autonòma de Barcelona –Universitat Politècnica de Catalunya –Universitat Pompeu Fabra –Universitat de Girona –Universitat de Lleida –Universitat Rovira i Virgili –Universitat Oberta de Catalunya –Biblioteca de Catalunya F India, Portugal, … F South Africa: SEALS, …

OhioLINK F Statewide Consortium F Represents 79 colleges, universities, libraries F Public Universities F Private Universities and Colleges F 2-Year Colleges F Only a few (e.g., Miami U. of Ohio) are also NDLTD members on their own

US University Members (44) F Penn. State University F Rochester Institute of Tech. F U. of Colorado Health Science Center F U. of Florida F U. of Georgia F University of Hawaii, Manoa F U. of Iowa F U. of Kentucky F U. of Maine F U. of North Texas – required since 8/99 F U. of Oklahoma F U. of South Florida F U. of Tennessee, Knoxville F U. of Tennessee, Memphis F U. of Texas at Austin – required in 2001 F U. of Virginia F U. Wisconsin - Madison F Vanderbilt U. F Virginia Commonwealth U. F Virginia Tech - required since 1/97 F West Virginia U. - required fall 1998 F Western Michigan U. F Worcester Polytechnic Inst. F Air University (Alabama) F Baylor University F Brigham Young University (part, whole) F Caltech F Clemson University F College of William & Mary F Concordia University (Illinois) F East Carolina University F East Tenn. State U. – require fall 2000 F Florida Institute of Technology F Florida International University F George Washington University F Louisiana State University F Marshall University (W. Va.) F Miami University of Ohio F Michigan Tech F Mississippi State University F MIT F Naval Postgraduate School (CA) F New Mexico Tech F North Carolina State University

Other Countries with Members F Belgium F Brazil F Canada F Germany F Hong Kong F India F Italy F Korea F Mexico F Netherland F Norway F Russia F Singapore F S. Africa F S. Korea F Spain F Taiwan F UK

SEALS (S. Africa) F Mellon Foundation $80K F South East Academic Library System F Border Technikon F Eastern Cape Technikon F University of Fort Hare F Port Elizabeth Technikon F Rhodes University (first to require outside US) F University of Port Elizabeth F University of Transkei F (and members elsewhere, e.g., University of Pretoria)

For professional societies F Like “writing across the curriculum”, e.g., Chemical Markup Language, MathML, … F Besides writing: computing/communications, information literacy, personal digital library management, tool use, research methods, collaboration, archiving/preservation F Data sets, communities of users of them F Classification systems / browsing / searching F NRC’s “On becoming a researcher”

Popular Works Liu, Xiangdong. Analysis and Reduction of Moire Patterns in Scanned Halftone Pictures (Ph.D. in Computer Science, May 1996; 6.6Mb) 7656 Petrus, Paul. Novel Adaptive Array Algorithms and Their Impact on Cellular System Capacity (Ph.D. in Electrical Engineering, March 1997; 5Mb) 2781 Agnes, Gregory Stephen. Performance of Nonlinear Mechanical, Resonant-Shunted Piezoelectric, and Electronic Vibration Absorbers for Multi-Degree-of-Freedom Structures (Ph.D. in Engineering Mechanics, Sept. 1997; ? Kb) 2492 Gonzalez, Reinaldo J. Raman, Infrared, X-ray, and EELS Studies of Nanophase Titania (Ph.D. in Physics, July 1996; 4607Kb) 1877 Shih, Po-Jen. On-Line Consolidation of Thermoplastic Composites (Ph.D. in Engineering Mechanics, Feb. 1997; 3.3Mb) 1791 Saldanha, Kevin J. Performance Evaluation of DECT in Different Radio Environments (MS in Electrical Engineering, Aug. 1996; 3.2Mb) 1431 DeVaux, David. A Tutorial on Authorware (MS in CS, April 1996; 2.3Mb) 1394 Kuhn, William B. Design of Integrated, Low Power, Radio Receivers in BiCMOS Technologies (Ph.D. in Electrical Engineering, Dec. 1995; 2518Kb)

International Use   United Kingdom  6082, Australia  Germany  Canada  South Korea  France  Italy  Netherlands  Brazil  Thailand  Greece

Networked University Digital Library (NUDL) F 1/15/99 NUDL proposal to NSF under DLI2 international program –VT: Library, Grad School, Industrial&Systems Eng. –Partners: UK (2), Singapore, Russia, Korea, Greece, Germany, plus Iberoamerican group (Spain, Portugal, Argentina, Brazil, Chile, Mexico) –Problems: Multilingual search, multimedia submissions, requirements/usability, … F Start with ETDs, then expand to other student works, portfolios, data sets, (CS) courseware,...

NUDL Partners F Ricardo A. Baeza-Yates, Universidad de Chile, Chile F José Luis Brinquete Borbinha, Biblioteca Nacional, Portugal F José Hilario Canós Cerdá, Universidad Politécnica de Valencia, Spain F Stavros Christodoulakis, Technical University of Crete, Greece F Eberhard Hilf, University of Oldenburg, Germany F Peter Diepold, Susanne Dobratz, …, Humboldt University, Germany F David Garza-Salazar, Monterrey Inst. of Technology (ITESM), Mexico F Sung Hyon Myaeng, Chungnam National University, Korea (+ colleagues) F Ana Maria Beltran Pavani, Prédio Cardeal Leme, Brazil F Lim Ee Peng, Nanyang Technological University, Singapore F Alexander I. Plemnek, St.-Petersburg State Technical University, Russia F Berthier Ribeiro-Neto, Federal U. of Minas Gerais (UFMG), Brazil F J. Alfredo Sánchez, Universidad de las Américas-Puebla, Mexico

Who are sponsors / cooperators? F Funding, Donations of hardware/software –SURA –US Dept. of Education (FIPSE) –Adobe Systems –IBM –Microsoft –OCLC F Others Serving on Steering Committee –National/Regional Projects: Australia, French speaking group (Montreal, Lyon), Germany, IberoAmerica (ISTEC, OAS), UK (UTOG) –CGS, National Lib. Canada, NSF, SOLINET, UMI, UNESCO,...

Relationship with publishers F Concern of faculty and students that still wish to publish books or journal articles, voiced: campus, Chronicle, NPR, Times F Solution: Approval Form gives students, faculty choices on access, when to change access condition; use IPR controls in DL F Solution: by case, work with publishers and publisher associations to increase access – AAP, AAUP – AAAS, ACM, ACS, Elsevier,...

Some responses from publishers F ACM: need to acknowledge copyright F Elsevier: need to acknowledge copyright F IEEE-CS: endorse initiative F ACS: After first publication, can release F Textbook publishers: different market, manuscript significantly reworked F General: restricting access to local campus will not cause any problems

How does this relate to UMI? F Generally, they are independent decisions. F 1987 UMI workshop was first to explore ETDs. F UMI wrote support letter for US Dept. of Ed. proposal. F UMI is on Steering Committee. F ProQuest Direct pilot of scanning works started 1/1/97, with free 2 yr access to front part. F We are collaborating on: – accepting electronic author submissions – standards (e.g., representation)

ETD Initiative (and UMI) Students Learn about DL, EPub TDs become more expressive N. Amer. (T)Ds are accessible, archived Global TDs become more accessible, archived UMI Universities

User Search Support (multilingual, XML) Note: All groups shown are connected with NDLTD.

F James Powell student project, D-Lib Magazine description in Sept F XML description of each site –type of search engine / service –language –coverage (for resource discovery) F Adding Z39.50 gateway capability and integrating with MARIAN, along with Harvest and Open Archives protocols

Access Approaches F Goal: Maximize access and services, e.g., by encouraging: F UMI centralized services F VTLS: planned free union collection of metadata F Distributed service: Dienst, Z39.50 F Regional services (e.g., OhioLink, AZ/NM) F Local servers with browse, search – From local catalogs to local archives F WWW robot indexing and search services

Access Possibilities Web search engines library catalog clients www. theses. org www. openarchives. org 3 rd Party Services (e.g., UMI) Virginia Tech National Library of Portugal CBUC (Spain) Ohio Link MITNational Projects: AU, GE, …

Why might a university want to be involved? F To improve graduate education / better prepare your students / increase their knowledge and visibility F To unlock university information F To save money for students and for the university / improve workflow F To build an important digital library

Multiple objectives F Sharing research results –Decrease costs, increase services –Increase knowledge of users F Adding to author knowledge/skills –Epub, DL, IPR F Enhancing organization’s infrastructure –CS department, library –University, Laboratory

How can a university get involved? F Select planning/implementation team – Graduate School – Library – Computing / Information Technology – Institutional Research / Educ. Tech. F Send us letter, give us contact names – F Adapt Virginia Tech solution – Build interest and consensus – Start trial / allow optional submission

Contact Our Project Team Phone Call Visit Video Tape

Convene Local Planning Group ETD

Build Local ETD Site Digital Library Policies Inspection/Approval Workshop/Training ETD

Responsibilities F Handle local education and collection –Contact information for helpers –Archive F Utilize standards –Metadata: MARC / DC-based concensus specification F Share metadata –Union services, mirrored services F Allow access – / –Open Archives Initiative (

Type 1 Members University Requires ETDs F Adobe Acrobat and/or XML/SGML tools F Automated submission & processing F Archive/access through UMI, (OCLC,) Virginia Tech,... F (Local) WWW site, publicity F (Local) Assistance provided as requested: , phone, listserv(s)

Type 2 Members University Agrees to Require ETDs F Like Type 1 but set date not reached F Usually has an option or pilot F May: wait for new AY; start with all who enter after; … F Build grass roots support – Advisory committee: representative? expert? – Champions to spread by word of mouth – Approval: Senates, Commissions, Deans, Students – Publicity to reach community

NDLTD Members, Types 3-7 F 3. Part of university requires ETDs F 4. University allows ETDs F 5. University investigating, has pilot F 6. University consortium joins: –CIC (Big 10 coordinating body) F 7. Non-university organization joins –CNI (Coalition for Networked Info.)

Support Services Developed F WWW site with > 300 Mb, CD, videotape F Automated submission system (MySQL, UNIX, WWW scripts - grad school/library) F Student guidelines, style sheets, multimedia training materials, FAQs, press info F SGML and XML DTDs for ETDs F SGML to HTML (web generator) F LaTeX, Word templates, converters F FTP site for PS to PDF conversion with UNIX distiller

Special Support F OCLC –Hosting Jan DC-XML metadata std meeting –Might help with preservation or Worldcat extract F VTLS –Machine, software, support staff to run free –Union catalog server: multilingual, MARC/DC-XML F UNESCO –ETD Guide in English, French, Spanish: book, Web –Scholarships to attend Int’l ETD Conference –Assistance with regional workshops: Columbia, …

Support Offered F Software, documentation, tech support F Listservs -stds, -UNESCO, …) F Evaluation: instruments, analysis - solutions/statistics F PetaPlex: temporary storage / archiving F MARIAN: research on a portable Java-based int’l service & archive –Multilingual –Gateway to Z39.50, Harvest, Open Archives –Local collections, harvested collections, federated search –Mediators/wrappers: MARC, XML, SGML, …

PetaPlex F Digital Library Machine (“super” object store): Parallel computer / storage utility F Research: inverted files, video server, … F Knowledge Systems Incorporated is supplying VT-PetaPlex-1 with 2.5 terabytes through 100 nodes : u Net connection + 25GB disk MHz Pentium + Linux

Future Work F Working with partners and publishers to increase level of access as much as possible F Interoperability tests among universities and with OAI to provide integrated services (+VTLS) F Study with testbed that emerges, to improve information retrieval, browsing, interface, and other types of user support F Evaluation, improving learning experience, spread to worldwide initiative, sustainable support and coordination