GSLIS Proseminar February 24, 2003 The Evolution of the Net: Predicting Network Infrastructure Bruce R. Schatz Graduate School of Library and Information Science University of Illinois at Urbana-Champaign (USA) Graduate School of Informatics Kyoto University (August 2001 – July 2002)

The Future of Infrastructure Niels Bohr on Quantum Theory “Prediction is very Difficult, especially about the Future”

What is Infrastructure? Infra means Inside Structure means Support Infrastructure is: Essential Services for Everyday Life Universal Protocols to Help Users Infrastructure Evolves to Deeper Levels New Technologies become Generic Enough

Outline of Talk The Evolution of Infrastructure Amateurs become Experts via Technology The Waves of the Net Data Information Knowledge The Laws of the Net The Basic Units of Evolution

Evolution of Infrastructure Everyone becomes An Expert with Interactive Support

The Structures of Everyday Life Bodies(individuals) Food and Clothes Buildings(groups) Houses and Cities Transportation(physical interactions) Rails (trains) and Roads (cars) Communication(logical interactions) Phones (talking) and Computers (retrieving)

How Infrastructure Scales Experts are Effective but Expensive Initially: All things done by Experts Eventually: Most things done by Amateurs Volume forces Provider Pyramid A few experts at the top A lot of amateurs at the bottom Infrastructure supports Ordinary People Everyone becomes a little bit of an Expert Most interactions done by amateurs

Example: Talking Transmission of Messages Generated by a person and Sent to another Network connects Operators to Operators The Telegraph of the 19 th Century Station to Station, trained professionals Operator transmits coded message The Telephone of the 20 th Century Person to Person, untrained amateurs Person speaks in ordinary words Infrastructure transmits coded message

Example: Retrieving Search of Archives Generated by many and Retrieved by many Network connects Archives to Users Bibliographic Search of the 20 th Century Station to Station, trained professionals Librarian queries central archives Web Search of the 21 st Century Person to Person, untrained amateurs Person types in ordinary words Infrastructure queries central archives

Provider Pyramids Emerge Telephone Network was Station-Station Person talked to Telephone Operator E.g. “get me a doctor!” Operator did manual switching to Person Volume forced Network to Person-Person In 1900, everyone must become an operator In 2000, everyone is an operator! Amateurs use directories then dial numbers Infrastructure supports Everyday Activity Automatic switching scales to volume

The Evolution of Infrastructure Experts handle Everyday Structures Provide services that become essential Increased Volume forces Provider Pyramid Amateurs handle most Interactions Infrastructure supports Amateurs Free Labor at Acceptable Quality Evolution proceeds by Copying Experts New Services approximate Experts Infrastructure adopts new Protocols Generic Technologies that always work

The Waves of the Net Network Infrastructure Evolves from Data to Information to Knowledge

What is the Net? The Net connects Everyone to Everything Global Network Infrastructure Universal Protocols to support Interactions The Functionality of the Net Evolves Data to Information to Knowledge Deeper Levels with Stronger Technologies At Any Level, the User only sees Outside Infrastructure is Inside, thus Invisible The Web is just the Present not the Future Someday, the Net will be true Cyberspace

Waves of the Net - Levels Data – The Past Transmission across Networks (Physical) Telephone Operators & Switching Gateways Information – The Present Retrieval across Spaces (Logical) Reference Librarians & Federated Repositories Knowledge – The Future Analysis across Models (Reality) Doctors & Categories Wisdom – The Goal

Waves of the Net - Services Data – The Past File Transfer Basic Unit is Bits within Packets Information – The Present Concept Navigation Basic Unit is Words within Documents Knowledge – The Future Structure Correlation Basic Unit is Features within Vectors

Waves of the Net -- Protocols Data – The Internet Packet Subwave: TCP/IP (Transmission) File Subwave: FTP (File Transfer Protocol) Information – The Interspace Document Subwave: HTTP/HTML (Search) Concept Subwave: CSP (Concept Switching) Knowledge – The Intermodel Vector Subwave: VTP/VCPL (Cluster Pattern) Structure Subwave: SMP (Structure Matching)

Transmitting Data Telephone Network of the 19 th Century Wave 0, Multiplexed Telegraph Digital Bits basic Syntax for Transmission Packets are the paradigm Packets are grouped Bits Address and Routing for Packets Bits interpreted only by local computers

The Internet First Universal Physical Wave Wave 1, Protocols across Networks Universal across packet networks (IP) Files from computer to computer Distributed File Systems Files cluster Bits within Computers Assemble Files from Bits using Packets (TCP) Transmit Files across Computers (FTP)

Searching Information WorldWide Web of the 20th Century Wave 2, Hyperlink Transmission (HTTP) Digital Words basic Semantics for Search Documents are the paradigm Documents are grouped Words (HTML) Classification and Indexing for Documents Words interpreted only by local repositories

The Interspace First Universal Logical Wave Wave 3, Protocols across Spaces Universal across document spaces Concepts from repository to repository Distributed Concept Systems Concepts cluster Words within Repositories Assemble Concepts from Words using Documents Search Concepts across Repositories

Universal Information Units Basic Concepts of Natural Language Fundamental Grouping needed for Semantics Specialized to specific Community Collections Essential Technologies all Converging Document Representation (XML, RDF) Language Parsing (multiword noun phrases) Statistical Indexing (contextual frequencies) Peer-Peer Networking (local computations) Concept Switching (vocabulary translations)

Same Process, Deeper Level The Interspace, using Libraries Subject thesaurus, yields Concept space Subject classification, yields Concept map From words to concepts From networks to spaces The Intermodel, using Lifestyles Reality problem, yields Structure model Reality solution, yields Structure cluster From features to structures From spaces to models

Analyzing Knowledge Lifestyle Grid of the 21 st Century Wave 4, Full-spectrum Questionnaires Digital Features basic Pragmatics for Analysis Vectors are the paradigm Vectors are grouped Features Comparison and Clustering for Vectors Features interpreted only by local situations

Healthcare Infrastructure Monitors for each Individual Capture everyday lifestyle status Customized questionnaires for healthcare Build detailed database of health vectors Managers for each Individual Daily Feature Record for each Individual Detailed Databases for whole Population Analyze Clusters of Similar Individuals

The Intermodel First Universal Representational Wave Wave 5, Protocols across Models Universal across vector models Features from individual to individual Distributed Structure Systems Structures cluster Features within Individuals Assemble Structures from Features using Vectors Analyze Structures across Individuals

Universal Knowledge Units Basic Structures of Everyday Life World Categories needed for Pragmatics Similar Structures from Many Subjects Medicine, categories of health status Computing, TCOL versus UNCOL Anthropology, cross-cultural universals Psychology, basic natural categories Architecture, design patterns for comfort life History, structures of everyday life

Realizing Wisdom Zen Buddhism The Ocean including all Waves Direct Interaction with Reality Beyond Words and Concepts Zen Koans Curriculum process of realization exercises Become “one with the koan”, achieve Satori Richi -- Reality Challenge, e.g. “Mu” Kikan – Reality Interaction, e.g. “Oak Tree”

The Intermind First Universal Spiritual Wave Wave N, Protocols across Minds Universal across Mind Realities Hive Mind on the Net, There is no Self! Religions to model Lifestyles richi (koan) for monitor (health) kikan (koan) for manager (health) Mimic basic structures not details Interactive navigation through automated clusters not automated decision-making

Metaphors across Waves Capturing Operators yields Internet Capturing Librarians yields Interspace Capturing Doctors yields Intermodel Capturing Roshi-s yields Intermind

The Laws of the Net Predicting Units of Network Infrastructure for Deeper Evolution

Philosophy of Informatics Physics in the 20 th Century Newtonian Mechanics is Western philosophy Deterministic Laws predict all motion Quantum Mechanics is Eastern philosophy Probabilistic Computations are uncertain Informatics in the 21 st Century Database Retrieval is Western philosophy Deterministic Rules match correct results Information Retrieval is Eastern philosophy Statistical Clusters suggest useful results

Laws of Motion Isaac Newton, 1687, Principia Mathematica I. Every body continues in its state of rest, or of uniform motion in a right [straight] line, unless it is compelled to change that state by forces impressed upon it. II. The change of motion is proportional to the motive force impressed; and is made in the direction of the right line in which that force is impressed. III. To every action there is always opposed an equal reaction; or, the mutual actions of two bodies upon each other are always equal, and directed to contrary parts.

Laws of Infrastructure Bruce Schatz, 2004, The Evolution of the Net I. Every infrastructure undergoes a characteristic evolution, from station-station to person-person, from trained experts to untrained amateurs, to handle the volume of a universal service. II. Most of the volume of a universal service is handled by the users themselves, with infrastructure support provided by modeling an appropriate expert. III. Each new basic structure models a new expert in terms of previous structures, and each successful evolution brings the structures closer to approximating reality.

Levels of Abstraction Deeper Levels drive Later Waves Each Wave has a fundamental level Deeper composed from Shallower Each Level built upon the Previous Level units level N-2 ordered by (ob) units level N-1 Context of N-2 within N-1 yields N Laws predict Future Units Information Space, Knowledge Model

Units of Abstraction Metaphors structures ob situations Situations categories ob structures Structures concepts ob categories Categories documents ob concepts Concepts terms ob documents Documents words ob terms

The Science of Informatics New Units from Old Units Predict Future Units from Current Units New Waves from Old Waves Predict Future Protocols from Current Same Processes, New Technologies Context as Meaning, Statistics as Truth Deeper Levels are Closer to Reality