Computing - The Next 10 Years Universal Access to Information Raj Reddy Carnegie Mellon University Pittsburgh, USA April 6, 2001 Talk presented at Georgia Tech 10 th Anniversary Convocation

Future Technology Computational power doubles every 18 months (Moore’s Law) Computational power doubles every 18 months (Moore’s Law) 100-fold improvement every 10 years 100-fold improvement every 10 years Disk Densities double every 12 months Disk Densities double every 12 months 1000-fold improvement every 10 years 1000-fold improvement every 10 years Optical bandwidth doubling every 9 months Optical bandwidth doubling every 9 months fold improvement every 10 years fold improvement every 10 years Infinite Bandwidth and Memory before Computation Infinite Bandwidth and Memory before Computation Cost decreasing, density increasing Cost decreasing, density increasing

What does the future hold? We can see some glimpses of the future Universities without walls, Universities without walls, Computers that never fail and self healing software Computers that never fail and self healing software Every home with giga PCs connected by gigabit networks Every home with giga PCs connected by gigabit networks Access to all the published creative works of the world Access to all the published creative works of the world anytime anywhere anyone anytime anywhere anyone Emergence of the World Bank of, not money, but Knowledge Emergence of the World Bank of, not money, but Knowledge Systems, so-called geriatric robotics, that help the disabled lead normal lives, and Systems, so-called geriatric robotics, that help the disabled lead normal lives, and Systems that give the rest of us superhuman capabilities, like getting a month’s work done in a day Systems that give the rest of us superhuman capabilities, like getting a month’s work done in a day

Universal Access to Information Information at your fingertips Access to all human knowledge: Access to all human knowledge: Anyone Anyone Anywhere Anywhere Anytime Anytime

All Human Knowledge Recorded Information Books Books Periodicals (journals, newspapers) Periodicals (journals, newspapers) Music, opera, dance Music, opera, dance Paintings, Sculptures and Monuments Paintings, Sculptures and Monuments Movies, video Movies, video Databases, software Databases, software Suppose all of this were on the Web

Examples from Lecture: Michael Shamos on UL Lecture: Michael Shamos on ULMichael Shamos on ULMichael Shamos on UL Books: A Child’s History of England Books: A Child’s History of EnglandA Child’s History of EnglandA Child’s History of England Art: Greek Art Art: Greek ArtGreek ArtGreek Art

Collection of static content Collection of static content Collection of dynamic multimedia content Collection of dynamic multimedia content Linearly organised Linearly organised Browsable, navigable Browsable, navigable Selected by an Author as related Selected by an Author as related Selected by User as related Selected by User as related Occupying a single physical location Occupying a single physical location No physical existence No physical existence Physically bound between cover Physically bound between cover Instantly Transmittable Instantly Transmittable What is a book? What is a digital book ?

What is a Library? What is a Library? Collection of items Collection of items Linearly organized (shelves) Linearly organized (shelves) Chosen by budget constraints Chosen by budget constraints Occupying physical space Occupying physical space Cataloged for access Cataloged for access

What is a Digital Library? What is a Digital Library? Collection of digital items Collection of digital items (potentially huge ) (potentially huge ) Encompassing everything (someday) Encompassing everything (someday) Organized arbitrarily Organized arbitrarily Occupying no physical space Occupying no physical space Fully content-searchable Fully content-searchable

Universal Library Implications Elimination of time, space, cost constraints Elimination of time, space, cost constraints Democratization of information Democratization of information “Knowledge is power” “Knowledge is power” Hyperlinks to related information Hyperlinks to related information Preservation and Dissemination of Knowledge Preservation and Dissemination of Knowledge faster and wider faster and wider Backup preservation Backup preservation Preservation of culture Preservation of culture

Universal Library Implications Research Research Web of scholarly information, reviews Web of scholarly information, reviews Teaching Teaching Support for distance education Support for distance education Academic publishing Academic publishing Virtual museums Virtual museums Interactivity Interactivity

Universal Library Applications Acess to “Born Digital” Information Acess to “Born Digital” Information World produces a Billion Billion(10 18 ) bytes of information every year(Lyman and Varian) World produces a Billion Billion(10 18 ) bytes of information every year(Lyman and Varian) 90% is stored digitally 90% is stored digitally Digital museum Digital museum Digital tour guide Digital tour guide What’s in the Taj Mahal? What’s in the Taj Mahal?

Universal Library Applications Research assistant Research assistant What did Newton write about color? What did Newton write about color? What are Moslem views on race? What are Moslem views on race? Teaching resource Teaching resource “Act out” books in virtual reality “Act out” books in virtual reality Real-time explanations Real-time explanations Business information Business information Data mining Data mining

We Can Store Everything 1 book = 500 pp. 1 book = 500 pp. 1MB uncompressed – 300KB compressed 1MB uncompressed – 300KB compressed 10 8 to 3x 10 8 books = ~10 14 bytes = 100 terabytes 10 8 to 3x 10 8 books = ~10 14 bytes = 100 terabytes Over 100 million computers on the Internet Over 100 million computers on the Internet At 1 GB each, >100 petabytes now At 1 GB each, >100 petabytes now 1 GB of disk costs ~$3 1 GB of disk costs ~$3 100 terabytes < $300 thousand to $1 million 100 terabytes < $300 thousand to $1 million

Non-textual Material 1 Movie = 10 GB 1 Movie = 10 GB 1 petabyte = 100,000 movies 1 petabyte = 100,000 movies All the movies ever made! All the movies ever made! Audio Audio 1 petabyte = 3000 years of music 1 petabyte = 3000 years of music All music ever performed or recorded All music ever performed or recorded Paintings and 1 MB Paintings and 1 MB 1 petabyte = 1 billion painting or photos 1 petabyte = 1 billion painting or photos

Non-textual Material Gore’s Digital Earth Gore’s Digital Earth “A multi-resolution, three-dimensional representation of the planet, into which we can embed vast quantities of geo-referenced data.” “A multi-resolution, three-dimensional representation of the planet, into which we can embed vast quantities of geo-referenced data.” Area of Earth  1/2 peta m 2 Area of Earth  1/2 peta m bytes/m 2 feasible 1000 bytes/m 2 feasible 2 MB/m 2 not practical yet  bytes = 1 zettabyte 2 MB/m 2 not practical yet  bytes = 1 zettabyte {peta-, exa-, zetta-, yotta-} {peta-, exa-, zetta-, yotta-}

Technological Challenges Input (scanning, digitizing, OCR) Input (scanning, digitizing, OCR) Data representation Data representation text, notations, images, web pages text, notations, images, web pages Navigation and Search Navigation and Search Multilingual Issues Multilingual Issues Output (voice, pictures, virtual reality) Output (voice, pictures, virtual reality) Synthetic Documents Synthetic Documents

Universal Library Design Modular Modular Technology plug-ins (e.g. machine translation) Technology plug-ins (e.g. machine translation) Distributed Distributed Mirror sites Mirror sites Multiple interfaces Multiple interfaces Human (languages, cultures, literacy) Human (languages, cultures, literacy) Machine Machine

Universal Library Design Speech input/output Speech input/output Pictorial output Pictorial output Language support Language support Translation assistants Translation assistants Summarization tools Summarization tools Synthetic documents Synthetic documents Encyclopedia-on-demand Encyclopedia-on-demand

Input Issues Non-digital media Non-digital media Conversion, scanning, correction Conversion, scanning, correction Triple keyboard, uncorrected OCR Triple keyboard, uncorrected OCR Digital media Digital media Formats, conversions, color representation Formats, conversions, color representation ASCII, HTML, SGML, XML, PDF, PS, TEX ASCII, HTML, SGML, XML, PDF, PS, TEX JPEG, TIFF, GIF? JPEG, TIFF, GIF?

Input Issues Structured matter Structured matter Musical notation, Laban Musical notation, Laban Chemistry Chemistry 3D Items 3D Items Resource allocation (what’s first?) Resource allocation (what’s first?) Duplication of effort (no registry) Duplication of effort (no registry)

Metadata Data about an item not part of the item Data about an item not part of the item Bibliographic Bibliographic Format, medium, encoding, resolution Format, medium, encoding, resolution Provenance Provenance Reliability, integrity Reliability, integrity Permissions Permissions Who generates metadata? Who generates metadata?

Navigation Browsing, finding, searching, flying Browsing, finding, searching, flying Fractal view Fractal view Keys are granularity and connectivity Keys are granularity and connectivity View whole collections or one glyph View whole collections or one glyph Understanding structure of information Understanding structure of information Making Sense Of The World’s Knowledge

Searching Mathematics

MATHEMATICA Canonical Form: Integrate[ Times[Power[E,Times[-1,Power[V1,2]]], Sin[Power[V1,2]]], {V1,0,Infinity}]

Multilingual Issues Character sets Character sets Representations Representations Íîäà ôèçè÷åñêè íàõîäèòñÿ â çäàíèè Èçâåñòèé Нода физически находится в здании Известий Multilingual navigation Multilingual navigation Translation assistance Translation assistance

Synthetic Documents Documents derived automatically from retrieved information Documents derived automatically from retrieved information Multilingual translation Multilingual translation Abstracts, summaries, glossaries Abstracts, summaries, glossaries Encyclopedia-on-demand Encyclopedia-on-demand

Information Reliability Existence  validity Existence  validity Universal Library Philosophy Universal Library Philosophy Avoid value judgments Avoid value judgments Provide information from which users (and programs) can assess validity Provide information from which users (and programs) can assess validity Source, reputation, recency, reviews, consistency Source, reputation, recency, reviews, consistency

Scaling Problems Search services (e.g. Altavista) index >10 8 documents Search services (e.g. Altavista) index >10 8 documents Suppose there were ? Suppose there were ? How can a billion users access the same item at once? How can a billion users access the same item at once?

Policy Challenges Use of copyrighted material Use of copyrighted material Economics (Who pays? Who gets?) Economics (Who pays? Who gets?) Privacy Privacy Reliability of information Reliability of information Change in the nature of teaching Change in the nature of teaching

Use Of © Content Philosophy: must pay for use Philosophy: must pay for use Authors, publishers will not suffer Authors, publishers will not suffer Implied license Implied license Automated permissions Automated permissions Bulk licensing Bulk licensing Compulsory licensing Compulsory licensing Owner CAN’T refuse; user MUST pay Owner CAN’T refuse; user MUST pay

Economics Flat-fee subscriptions (e.g. HBO) Flat-fee subscriptions (e.g. HBO) Metered use (electric company) Metered use (electric company) Microcharge (Tobias “clickl”) Microcharge (Tobias “clickl”) Free (paid by government) Free (paid by government) Automated permissions Automated permissions Use measured by technology Use measured by technology

Operating Model Single portal for access to all information Single portal for access to all information Universal Library provides input, access, multilingual, output and synthesis tools Universal Library provides input, access, multilingual, output and synthesis tools Universal Library will be a model scanning operation Universal Library will be a model scanning operation Registry of digitized works Registry of digitized works

Operating Model Specialized collections curated by specialists, provided to Universal Library Specialized collections curated by specialists, provided to Universal Library Foreign collection performed in foreign countries Foreign collection performed in foreign countries Universal Library will be mirrored in ~12 sites around the world Universal Library will be mirrored in ~12 sites around the world

Universal Library Status >13,000 digital volumes >13,000 digital volumes Art Art Newspapers Newspapers Music, video Music, video Portal to hundreds of other collections Portal to hundreds of other collections Visit Visit

Projects Navigator Navigator Academic electronic publishing Academic electronic publishing Electronic Union Catalog Electronic Union Catalog Books out of copyright books out of print Books out of copyright books out of print Software distribution Software distribution

Conclusions and Recommendations Conclusions Conclusions Barely 10% of all public information is available on the Internet Barely 10% of all public information is available on the Internet Government needs to play a leadership role in developing digital libraries Government needs to play a leadership role in developing digital libraries Significant technical and operational challenges in migrating and maintaining holdings in digital form Significant technical and operational challenges in migrating and maintaining holdings in digital form Intellectual Property rights need to be addressed to facilitate creation and access digital libraries Intellectual Property rights need to be addressed to facilitate creation and access digital libraries Recommendations Recommendations Support research: meta data, scalability, multiple languages, security, and usability Support research: meta data, scalability, multiple languages, security, and usability Create testbeds: million book project Create testbeds: million book project Place all public governmental information online Place all public governmental information online Preserve IP rights of creators by creating tax incentives for public use of online copyrighted information Preserve IP rights of creators by creating tax incentives for public use of online copyrighted information