1. Involving New Scholars in Digital Libraries through the Networked Digital Library of Theses and Dissertations (NDLTD) Telcordia, Morristown, NJ December 30, 1999 Edward A. Fox fox@vt.edu CC CS DLRL Internet TIC Virginia Tech, Blacksburg, VA, USA

2. Acknowledgements (Selected) F Sponsors: ACM, Adobe, IBM, Microsoft, NSF, OCLC, US Dept. of Education, … F Co-PIs: Marc Abrams, Robert Akscyn, John Carroll, John Eaton, Gail McMillan F Students: Fernando Das Neves, Robert France, Marcos Goncalves, Neill Kipp, Paul Mather, Constantinos Phanouriou, James Powell, Ohm Sornil, David Watkins, Chang Zhang, Jianxin Zhao, … F Information Systems at Virginia Tech – see online full version of short introduction to networking efforts, at http://rdweb.cns.vt.edu/talks/VT-Initiatives-9-99.ppt (by Jeff Crowder and Erv Blythe)

3. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

4. Virginia Tech Background F Largest university in Virginia, land-grant, town population 35K plus 25K students, #2 in football F Blacksburg Electronic Village, since 1992, with 80% of community on Internet F Net.Work.Virginia, largest ATM network, with over 600 sites, for education, research, govt F LMDS, Local Multipoint Distribution Service, gigabit wireless networking - 1/3 of Virginia F Math Emporium, 500 workstations F Faculty Development Initiative, round 2

5. Virginia Tech CS F Department of CS focused on HCI since 1994 F $2M (NSF RI) labs: usability, group decisions, info access F Faculty (+ Kafura – OO/real-time, Head) –Abrams (Network Research Group, UIML – user interface) –Barfield (ISE - wearable) –Bowman (virtual environments and interface issues) –Carroll (design, scenarios, education, BEV) –Ehrich (equipment, graphics, BEV) –Hartson (theory & methodology, remote evaluation) –Hix (usability, VR/CAVE) –Ramakrishnan (data mining, recommemder systems) –Rosson (object orientation/languages, collaboration) –Shaffer (problem solving environments, education, GIS) –Williges (ISE - experimentation, meta-evaluation)

6. ACITC F Advanced Communications and Information Technology Center, opening summer 2000 F Connects to the library, with a focus on IT F 1/3 high-tech (multimedia) classrooms F 1/3 digital/electronic library (reading room) F 1/3 research labs: 10, including: –Digital Library Research Laboratory (DLRL) –Center for Applied Technologies in the Humanities –HCI; HPC; Multimedia; Visualization (CAVE), … –Spaces for industry-supported labs, visitors

7. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

8. Definition: Digital Libraries are complex systems that F help satisfy info needs of users (societies) F provide info services (scenarios) F organize info in usable ways (structures) F present info in usable ways (spaces) F communicate info with users (streams)

9. 5S Layers Societies Scenarios Spaces Structures Streams

10. Definition: 5S Framework F Societies: interacting people (, computers) F Scenarios: services, functions, operations, methods F Spaces: domains + constraints (e.g., distance, adjacency): 2D, vector, probability F Structures: relations, trees, nodes and arcs F Streams: sequences of items (text, audio, video, network traffic) F (5 Element System: Fire, Wood, Earth, Metal, Water)

11. 5S: Components F Societies: roles, rituals, reasons, relationships, artifacts F Scenarios: acquire, index, consult, administer, preserve F Spaces: physical, temporal, functional, presentational, conceptual F Structures: architectures, taxonomies, schema, grammars, links, objects F Streams: granularities, protocols, paths, flows, turbulences

12. Neill Kipp Dissertation F Training interested groups about 5S and the Star Methodology, refining the Framework to have solid mathematical foundation F Case studies of projects at Virginia Tech or involving VT staff/students: CSTC, NDLTD, NARA (National Archives, with SAIC), Lexis,... F Open also to study DL projects elsewhere F Focusing too on the design artifacts developed and related issues of efficient description and representation (esp. with markup, hypermedia)

13. Digital Libraries --- Virginia Tech F MARIAN (NLM) F CS DL Prototype - ENVISION (NSF, ACM) F TULIP (Elsevier, OCLC) F BEV History Base (NSF, Blacksburg) F DL for CS Education - EI (NSF, ACM) F WATERS, NCSTRL (NSF) F NDLTD (SURA, US Dept. of Education) F CSTC (NSF, ACM), CRIM (NSF, SIGMM) F WCA (Log) Repository (W3C) F VT-PetaPlex-1 (Knowledge Systems)

14. Digital Library Courseware F http://ei.cs.vt.edu/~dlib/ F WWW pages or large PDF copy files F Online quizzes based on book by Michael Lesk (Morgan Kaufmann Publishers) F Contents based on book, with several other popular topics added (e.g., agents) F Separate pages to supplement: Definitions, Resources (People, Projects), and References

15. CS -> CSTC -> CRIM F NSF and ACM Education Committee are funding a 2 year project “A Computer Science Teaching Center” - CSTC - http://www.cstc.org/ F College of NJ, U. Ill. Springfield, Virginia Tech F Focus initially on labs, visualization, multimedia F Multimedia part is also supported by a 2nd grant to Virginia Tech and The George Washington University: http://www.cstc.org/~crim/ (with curricular guidelines also under development) F ACM will help provide reward for contributors, through Journal of Computing Education: Resources and Research (JoCERR?)

16. Browsing (2)

17. DLs Shorten the Chain from Editor Publisher A&I Consolidator Library Reviewer

18. DLs Shorten the Chain to Author Reader Digital Library Editor Reviewer Teacher Learner Librarian

19. Enhancing Learning with DLs

20. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

21. A Digital Library Case Study F Domain: graduate education, research F Genre:ETDs=electronic theses & dissertations F Submission: http://etd.vt.edu F Collection: http://www.theses.org Project: Networked Digital Library of Theses & Dissertations (NDLTD) http:// www.ndltd.org

22. Media ETD Web Site http://www.ndltd.org/ ETDs Got Your Interest? Graduate Students Singapore AM Chronicle of Higher Ed. National Public Radio NY Times... U. Laval

23. Key Ideas: Networked infrastructure Scalability Education is the rationale University collaboration Workflow, automation Authors must submit Maximal access PDF, SGML, MM Standards Federated search 8th graders vs. grads MARC, DC, URNs

24. F Aiding universities to enhance grad educ., publishing and IPR efforts F Helping improve the availability and content of theses and dissertations F Educating ALL future scholars so they can publish electronically and effectively use digital libraries (i.e., are Information Literate and can be more expressive) What are we doing?

25. What are the long term goals? F 400K US students / year getting grad degrees are exposed / involved F 200K/yr rich hypermedia ETDs that may turn into electronic portfolios F Dramatic increase in knowledge sharing: lit. reviews, bibliographies, … F Services providing lifelong access for students: browse, search, prior searches, citation links

26. What led to today’s meeting? F 1987 mtg in Ann Arbor: UMI, VT, … F 1992 mtg in Washington: CNI, CGS, UMI, VT and 10 universities with 3 reps each F 1993 mtg in Atlanta to start Monticello Electronic Library (MEL): SURA, SOLINET F 1994 mtg in Blacksburg re ETD project: std of PDF + SGML + multimedia objects F 1996 funding by SURA, US Dept. of Education (FIPSE) for regional, national projects F 1997 meetings in UK, Germany,... F Sept. 1999 meeting in Paris at UNESCO Headquart. http://www.unesco.org/webworld/etd/

27. Status of the Local Project F Approved by university governance Spring 1996; required starting 1/1/97 F Submission & access software in place F Submission workshops for students (and faculty) occur often: beginner/adv., focused on Adobe PDF and multimedia formats F Faculty training as part of Faculty Development Initiative F Over 2000 ETDs in collection

28. Library Costs F $12/vol. for paper thesis processing – catalog, bind, security strip, label, and shelving – @950 vols./yr. = $11.4K F $3.20/vol. ETD processing – cataloging @950 vols./yr. = $3040 F $.07/vol. Shelving (save 166 ft/yr) F $.04/vol. Circulation (of 3000 copies/yr)

29. NDLTD Computer Resources Research Literature Student Prepares Thesis or Dissertation

30. Student Defends and Finalizes ETD My Thesis ETD

32. Student Gets Committee Signatures and Submits ETD Signed Grad School

33. Graduate School Approves ETD Student is Graduated Ph.D.

34. Library Catalogs ETD and New Students Have Access to the New Research WWW NDLTD

35. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

36. Institutional Members F Coalition for Networked Information (CNI) F Committee on Institutional Cooperation (CIC) F Diplomica.com F Dissertation.com F Dissertationen Online (Germany) F Ibero-American Science & Technology Education Consortium (ISTEC, www.istec.org) F National Library of Portugal (for all universities) F Organization of American States (SEDI/OAS) F UNESCO (www.unesco.org/webworld/etd)

37. US University Members (35+) F Rochester Institute of Tech. F U. of Colorado Health Sci. Cntr. F U. of Florida F U. of Georgia F University of Hawaii, Manoa F U. of Iowa F U. of Maine F U. of Oklahoma F U. of South Florida F U. of Tennessee, Knoxville F U. of Tennessee, Memphis F U. of Texas at Austin F U. of Virginia F U. Wisconsin - Madison F Vanderbilt U. F Virginia Tech - required since 1/97 F West Virginia U. - required fall 1998 F Worcester Polytechnic Inst. F Air University (Alabama) F Brigham Young University F Cal Tech F Clemson University F College of William & Mary F Concordia University (Illinois) F East Carolina University F East Tenn. State University F Florida Institute of Tech. F Florida International University F George Washington University F Marshall University (W. Va.) F Miami U. of Ohio F MIT (in process) F Michigan Tech F Naval Postgraduate School (CA) F North Carolina State U. F Penn. State University

38. Australian Project Members F U. New South Wales (lead institution) F U. of Melbourne F U. of Queensland F U. of Sydney F Australian National University F Curtin U. of Technology F Griffith U.

39. German Project Members F Humboldt University (lead institution) F 3 other universities F 5 learned societies –Mathematics, Physics, Chemistry, Sociology, Education F 1 computing center F 2 major libraries

40. CBUC (www.cbuc.es, Spain) F Consorci de Biblioteques Universitàries de Catalunya, as group, with 9 members: –Universitat de Barcelona –Universitat Autonòma de Barcelona –Universitat Politècnica de Catalunya –Universitat Pompeu Fabra –Universitat de Girona –Universitat de Lleida –Universitat Rovira i Virgili –Universitat Oberta de Catalunya –Biblioteca de Catalunya

41. Other International Members F Chinese University of Hong Kong F Chungnam National U., Dept of CS (S. Korea) F City University, London (UK) F Darmstadt U. of Tech. (Germany) F Free University of Berlin (Germany - Vet. Med.) F Gyeongsang National U. (Korea) F India Institute of Technology, Bombay (India) F Nanyang Technological U. (Singapore, part) F National U. of Singapore (Singapore, part) F Polytechnic University of Valencia (Spain) F Rhodes U. (South Africa) F St. Petersburg St. Tech.U (Russia) F Univ. de las Américas Puebla (Mexico) F Univ. of Alicante (Spain) F Univ. of Pisa (Italy) F U. Laval; U. of Guelph; U. Waterloo; Wilfrid Laurier U. (Canada)

43. Popular Works 1996 458 Seevers, Gary L. Identification of Criteria for Delivery of Theological Education Through Distance Education: An International Delphi Study (Ph.D., Educational Research and Evaluation, April 1993; 1353Kb) 432 Hohauser, Robyn Lisa. The Social Construction of Technology: The Case of LSD (MS in Science and Technology Studies, Feb. 1995; 244Kb) 390 Childress, Vincent William. The Effects of Technology Education, Science, and Mathematics Integration Upon Eighth Grader's Technological Problem-Solving Ability (Ph.D. in Vocational and Technical Education, July 1994; 285Kb) 310 Kuhn, William B. Design of Integrated, Low Power, Radio Receivers in BiCMOS Technologies (Ph.D. in Electrical Engineering, Dec. 1995; 2Mb) 287 Sprague, Milo D. A High Performance DSP Based System Architecture for Motor Drive Control ( MS in Electrical Engineering, May 1993; 878Kb) 165 Wallace, Richard A. Regional Differences in the Treatment of Karl Marx by the Founders of American Academic Sociology (MS in Sociology, Nov. 1993; 479Kb) 150 McKeel, Scott Andrew. Numerical Simulation of the Transition Region in Hypersonic Flow (Ph.D. in Aerospace Engineering, Feb. 1996; 3Mb)

44. Popular Works 1997 9920 Liu, Xiangdong. Analysis and Reduction of Moire Patterns in Scanned Halftone Pictures (Ph.D. in Computer Science, May 1996; 6.6Mb) 7656 Petrus, Paul. Novel Adaptive Array Algorithms and Their Impact on Cellular System Capacity (Ph.D. in Electrical Engineering, March 1997; 5Mb) 2781 Agnes, Gregory Stephen. Performance of Nonlinear Mechanical, Resonant-Shunted Piezoelectric, and Electronic Vibration Absorbers for Multi-Degree-of-Freedom Structures (Ph.D. in Engineering Mechanics, Sept. 1997; ? + 7926Kb) 2492 Gonzalez, Reinaldo J. Raman, Infrared, X-ray, and EELS Studies of Nanophase Titania (Ph.D. in Physics, July 1996; 4607Kb) 1877 Shih, Po-Jen. On-Line Consolidation of Thermoplastic Composites (Ph.D. in Engineering Mechanics, Feb. 1997; 3.3Mb) 1791 Saldanha, Kevin J. Performance Evaluation of DECT in Different Radio Environments (MS in Electrical Engineering, Aug. 1996; 3.2Mb) 1431 DeVaux, David. A Tutorial on Authorware (MS in CS, April 1996; 2.3Mb) 1394 Kuhn, William B. Design of Integrated, Low Power, Radio Receivers in BiCMOS Technologies (Ph.D. in Electrical Engineering, Dec. 1995; 2518Kb)

45. Usage of ETDs in VT Collections 1996199719981999 Jan-Aug Total requests 37,171247,537465,974907,104 Daily Requests 1026851,7223,121 Abstract requests 25,829112,633177,647143,056 Hosts served 9,01522,72528,02252,663

46. International Use  199619971998  85029928170 United Kingdom  6082,5014223 Australia  34623787373 Germany  71323673970 Canada  38712642201 South Korea  46311614431 France  2507252553 Italy  1918672781 Netherlands  18311301449 Brazil  229671089 Thailand  839581414 Greece

48. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

49. Relationship with publishers F Concern of faculty and students that still wish to publish books or journal articles, voiced: campus, Chronicle, NPR, Times F Solution: Approval Form gives students, faculty choices on access, when to change access condition; use IPR controls in DL F Solution: by case, work with publishers and publisher associations to increase access – AAP, AAUP – AAAS, ACM, ACS, Elsevier,...

50. Some responses from publishers F ACM: need to acknowledge copyright F Elsevier: need to acknowledge copyright F IEEE-CS: endorse initiative F ACS: After first publication, can release F Textbook publishers: different market, manuscript significantly reworked F General: restricting access to local campus will not cause any problems F Survey by Joan Dalton, Canada

51. For professional societies F Like “writing across the curriculum” F Besides writing: computing/communications, information literacy, personal digital library management, tool use, research methods, collaboration, archiving/preservation F Data sets, communities of users of them F Classification systems / browsing / searching F National Research Council (NRC) booklet “On becoming a researcher in the digital age”

52. Who are sponsors / cooperators? F Funding, Donations of hardware/software –SURA –US Dept. of Education (FIPSE) –Adobe Systems –IBM –Microsoft –OCLC F Others Serving on Steering Committee –National/Regional Projects: Australia, French speaking group, Germany, IberoAmerica (ISTEC), UK (UTOG) –Council of Graduate Schools, National Lib. Canada, NSF, OAS, SOLINET, UMI, UNESCO,...

53. How does this relate to UMI? (Bell and Howell) F 1987 UMI workshop to explore ETDs F Support letter for US Dept. of Ed. proposal F Steering and technical committee membership F ProQuest Direct pilot of scanning works started 1/1/97, free 2 yr access to front part F Collaborating on: – accepting electronic author submissions – standards (e.g., representation), research

54. ETD Initiative (and UMI) Students Learn about DL, EPub TDs become more expressive N. Amer. (T)Ds are accessible, archived Global TDs become more accessible, archived UMI Universities

55. How can a university get involved? F Select planning/implementation team – Graduate School – Library – Computing / Information Technology – Institutional Research / Educ. Tech. F Send us letter, give us contact names F Adapt Virginia Tech solution – Build interest and consensus – Start trial / allow optional submission

56. Convene Local Planning Group ETD

57. Build Local ETD Site Digital Library Policies Inspection/Approval Workshop/Training ETD

58. Type 1 Members University Requires ETDs F Adobe Acrobat and/or XML/SGML tools F Automated submission & processing F Archive/access - UMI, (OCLC, Center for Research Libraries, Virginia Tech,)... F (Local) WWW site, publicity F (Local) Assistance provided as requested: email, phone, listserv(s)

59. Type 2 Members University Agrees to Require ETDs F Like Type 1 but set date not reached F Usually has an option or pilot F May: wait for new AY; start with all who enter after; … F Build grass roots support – Advisory committee: representative? expert? – Champions to spread by word of mouth – Approval: Senates, Commissions, Deans, Students – Publicity to reach community

60. NDLTD Members, Types 3-7 F 3. Part of university requires ETDs F 4. University allows ETDs F 5. University investigating, has pilot F 6. University consortium joins: –CBUC (Catalunya, Spain) F 7. Non-university organization joins –CNI (Coalition for Networked Info.) –ISTEC, OAS, UNESCO, …

61. Everyone Learns F Students become “info literate” F Students learn about discovery, search, categorization/classification (e.g., CoRC), e-pub (e.g., XML, multimedia, hypertext), preservation, helping others find/reuse F Campus starts to think about IPR –e.g., Virginia Tech symposium http://www.rgs.vt.edu/resmag/seminars.html F Faculty and students improve quality as reader base expands

62. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

63. User Search Support (multilingual, XML) Note: All groups shown are connected with NDLTD.

64. www.theses.org F James Powell student project, D-Lib Magazine description in Sept. 1998 F XML description of each site –type of search engine / service –language –coverage (for resource discovery) F Adding Z39.50 gateway capability and integrating with MARIAN, along with Harvest and Open Archives protocols (according to Santa Fe Convention) – see www.openarchives.org

65. Open Archives Initiative F Santa Fe meeting, Oct. 21-22, 1999 F Workshop early June, San Antonio, DL’00 F LANL, CNI, DLF, Mellon, … F Convention F Archives -> Open Archives –Support unique archive identifiers –Implement Open Archives Metadata Set (DC-based, using XML) –Implement Dienst harvesting interface –Register the archive F Build tools, layer other services: linking, searching, …

66. Approaches to Open Archives Build By discipline Build By institution Author Category Interdisciplinary Year Language Query …

67. Open Archives Members  Original Participants in the Open Archives Initiative – Caroline Arms, Library of Congress – Leslie Carr, University of Southampton – Mark Doyle, American Physical Society – Dale Flecker, Harvard University – Edward A. Fox, Virginia Tech – Michael Friedman, HighWire Press, Stanford University – Paul M. Gherman, Vanderbilt University – Paul Ginsparg, Los Alamos National Laboratory & xxx – Stevan Harnad, University of Southampton – Thomas Krichel, University of Surrey & RePEc – Carl Lagoze, Cornell University – Rick Luce, Los Alamos National Laboratory – Clifford Lynch, Coalition for Networked Information – Kurt Maly, Old Dominion University – Michael L. Nelson, NASA Langley Research Center – John Ober, California Digital Library – Bob Parks, Washington University & EconWPA – Herbert Van de Sompel, University of Ghent – Eric F. Van de Velde, California Institute of Technology – Don Waters, The Andrew W. Mellon Foundation – Ken Weiss, California Digital Library  Others Joining (selected) – University of Virginia – Jim French, Worthy Martin, Thornton Staples, – NEC Research Institute - C. Lee Giles and Steve Lawrence – Internet Archive - Kurt Bollacker, Marlita Kahn – India - University of Mysore – Shalini Urs – Mexico – University of Monterrey - David Garza Salazar

68. Access Approaches F Goal: Maximize access and services, e.g., by encouraging: F UMI centralized services F Distributed service: Dienst, Z39.50, … F Regional services (e.g., OhioLink) F Global service: Open Archives F Local servers with browse, search – From local catalogs to local archives F WWW robot indexing and search services

69. Access Possibilities Web search engines library catalog clients www. theses. org www. openarchives. org 3 rd Party Services (e.g., Bell & Howell) Virginia Tech National Library of Portugal CBUC (Spain) Ohio Link MITNational Projects: AU, GE, …

70. Support Services Developed F WWW site with > 300 Mb, CD, videotape F Automated submission system (MySQL, UNIX, WWW scripts - grad school/library) F Student guidelines, style sheets, multimedia training materials, FAQs, press info F SGML and XML DTDs for ETDs F SGML to HTML (web generator) F LaTeX, Word templates, converters

71. Support Offered F Software, documentation, tech support F Email, listservs (etd-l@listserv.vt.edu, - eval, -grad, -library, -technical) F Donations: Adobe, Microsoft F Evaluation: instruments, analysis http://scholar.lib.vt.edu - solutions/statistics F (Temporary storage / archiving; aid - in setting up an int’l service & archive)

72. Enhancements F Dublin Core spec, MARC crosswalk F DTDs for SGML, XML(+ ML) F Annotation system (author, friends, notes) F Routing system (based on Sift) F Multilingual WWW site, training materials F Better federated search (w. Z39.50, planned with Dienst and Harvest - using MARIAN) F Integrate SFX, CiteSeer (citation database and linking, plagiarism detection)

73. Accessibility Activities / Plans F Interface design (simple, 3D, VR) F Usability studies F Generic multi-lingual support F Support for those with disabilities F Hybrid collection (paper, MARC, abstracts, full-text, multimedia) F Disciplinary classifications, tools F Visualization of results, collection

74. SPIRE Visualization

75. CAVE Experiments F Use a familiar metaphor –building / floor / room / shelf / book F Rearrange orderings / shelving –use categories, clustering, ranking –use visualization: colors and gaps –study space mappings: physical, logical F Simplify movement for key tasks

77. ENVISION F NSF “A User-Centered Database from the Computer Science Literature” (1991-93) F Collected bib/typesetter data, converted to SGML F Scanned thousands of page images F MARIAN search engine - can be made available (also applied to the Virginia Tech library catalog) used as part of a prototype object-based DL, with tailored visualization interface (L. Nowell dissertation)

78. Envision Results Window

79. MARIAN F Multiple Access Retrieval of Information with ANnotations F (Musical: Marian the Librarian …) F Evolved from 1980’s CODER system to a distributed Online Public Access Catalog (OPAC), then DL backend, now becoming a full DL system F From C/C++ to Java by Jianxin Zhao F Future uses: NDLTD, NUDL, PetaPlex

82. MARIAN Layers Database Layer Search Engine Layer User Information Layer User Interface Layer User

84. MARIAN Testing Architecture Load Generator Webgate Java Server C/C++ Server

85. MARIAN Parallelism

86. MARIAN Response Time

87. France Dissertation F Key developer since CODER F Applying computational linguistics efforts with machine readable dictionaries F Applying opportunistic handling of term lists for ranking, usable displays (“to be or not to be, that is the”) F Developing and evaluating variety of interfaces

88. PetaPlex F Digital Library Machine (“super” object store) F Parallel computer / storage utility for scale of 1000 to 100,000,000 gigabytes (1 Tbyte - 100 Pbyte) F Knowledge Systems Incorporated supplied VT- PetaPlex-1 for $250,000 with –high speed backbone connections (eventually 1 Gbps) –2.5 terabytes through 100 “Nanoservers”: –Each = Network connection + IBM 25GB disk + 233 MHz Pentium II + Linux

89. PetaPlex Complex FRONT END MACHINE RS/6000, 1G RAM, 4 Proc. Nanoserver Service Machine 1 Service Machine 2 Service Machine 3 Service Machine 4

90. PetaPlex Service Machines F Front-end provides handle/repository abstraction through hashing F Small object server F Large object server –video on demand –streaming audio F Information retrieval server F Proxy / cache server (e.g., 1 terabyte server of 1000 worldwide for Comsat/Intelsat)

91. PetaPlex Top View 4 ft. side

92. PetaPlex Side View 4 ft. wide 8 ft. high Roles: * Support * Cooling * Power 15 shelves

93. PetaPlex Cost Goals, Approach F Maximize number of seeks achievable F Maximize % of cost invested in disks F Maximize flexibility and reliability F Minimize cost per unit of storage F Approach “information utility” F Increase throughput and reliability by replicating on other PetaPlex systems F Use robotics, wireless, and commodity production of nanoservers

94. Sornil & Mather Dissertations F Proposing 100 Tbyte wireless Petaplex for $2M F Mather: efficiently handling very large numbers of objects of varying sizes F Sornil: efficiently handling IR for very large dynamic collections, large numbers of users, high transaction rates, large inverted files –modeling and simulation –data organization –parallelization of algorithms, alone and in combination for retrieval (related) tasks

95. Problems Addressed T HE P ETA P LEX -T YPE A RCHITECTURE

96. Early Results PARTITIONING SCHEMES: A PRELIMINARY ANALYSIS  To preliminarily study effects of three important parameters on the performance of the partitioning schemes  Term selection characteristic  Number of queries in the system  Number of disk nodes  To have a preliminary performance trend of the Hybrid partitioning scheme  Inverted File Partitioning Schemes:  Term Partitioning  Document Partitioning  Hybrid Partitioning

97. Early Results Effects of Number of Disk Nodes Skewed Term Selection (with 1024 queries in the system) Uniform Term Selection (with 1024 queries in the system)

98. OUTLINE F Introduction F Digital libraries F NDLTD case study F Members, statistics F Relationships, universities F Access, software, hardware F Conclusion

99. DL Challenges F Preservation - so people with trust DLs F Supporting infrastructure – affordable storage in large capacity, very fast networks,... F Scalability, sustainability, interoperability F DL industry - critical mass by covering libraries, archives, museums, corporate info, govt info, personal info - “quality WWW” integrating IR, HT, MM,... –Need tools & methods to make them easier to build