1. The Endeavour Expedition: Charting the Fluid Information Utility
Randy H. Katz, Principal Investigator
EECS Department
University of California, Berkeley
Berkeley, CA

2. Why “Endeavour”? DARPA BAA 99-07: Information Technology Expeditions
To strive or reach; a serious determined effort (Webster’s 7th New Collegiate Dictionary); British spelling
Captain Cook’s ship from his first voyage of exploration of the great unknown of his day: the southern Pacific Ocean ( ).
These voyages brought brought more land and wealth to the British Empire than any military campaign.
Cook’s lasting contribution: comprehensive knowledge of the people, customs, and ideas that lay across the sea
“He left nothing to his successors other than to marvel at the completeness of his work.”

3. Expedition Goals
Enhancing human understanding through information technology
Dramatically more convenient for people to interact with information, devices, and other people
Supported by a “planetary-scale” Information Utility
Stress tested by challenging applications in decision making and learning
New methodologies for design, construction, and administration of systems of unprecedented scale and complexity
Figure of merit: how effectively we amplify and leverage human intellect
A pervasive Information Utility, based on “fluid systems technology” to enable new approaches for problem solving & learning

4. Expedition Assumptions
Human time and attention, not processing or storage, are the limiting factors
Givens:
Vast diversity of computing devices (PDAs, cameras, displays, sensors, actuators, mobile robots, vehicles); No such thing as an “average” device
Unlimited storage: everything that can be captured, digitized, and stored, will be
Every computing device is connected in proportion to its capacity
Devices are predominately compatible rather than incompatible (plug-and-play enabled by on-the-fly translation/adaptation)

5. Expedition Challenges
Personal Information Mgmt is the Killer App
Not corporate processing but management, analysis, aggregation, dissemination, filtering for the individual
People Create Knowledge, not Data
Not management/retrieval of explicitly entered information, but automated extraction and organization of daily activities
Information Technology as a Utility
Continuous service delivery, on a planetary-scale, on top of a highly dynamic information base
Beyond the Desktop
Community computing: infer relationships among information, delegate control, establish authority

6. Expedition Approach Information Devices Information Utility
Beyond extrapolated desktop devices to MEMS-sensors/actuators plus capture/display to yield enhanced activity spaces
Information Utility
“Fluid”, Network-Centric System Software
Paths/Streams: process/store/manage information
“Movable” Processing and Storage
Partitioned/distributed functionality Thin-Clients/Fat-Infrastructure
Nomadic Data
Negotiation-based Interfaces
Always-Available Functionality
Wide-area distributed coordination and control on scalable servers

7. Expedition Approach Information Applications Design Methodology
High Speed/Collaborative Decision Making, Learning
Augmented “Smart” Spaces: Rooms, Vehicles
Design Methodology
HW/SW Co-design
Formal Methods
Decomposable and Reusable Components
User-centered Design

8. Applications Collaboration Spaces Info Appliances Information Utility
High Speed
Decision Making
Learning
Classroom
E-Book
Vehicles
Applications
Collaboration Spaces
Info Appliances
Human Activity Capture
Generalized UI Support
Event Modeling
Transcoding, Filtering, Aggregating
Statistical Processing/Inference
Proxy Agents
Negotiated APIs
Self-Organizing Data
Interface Contracts
Wide-area Search & Index
Information
Utility
Nomadic Data & Processing
Wide-Area Data & Processing
Automated Duplication
Movement & Positioning
Distributed Cache Management
Stream- and Path-Oriented Processing & Data Mgmt
Non-Blocking RMI
Soft-/Hard-State Partitioning
PDA
Laptop
Wallmount Display
Camera
Information
Devices
Handset
Smartboard
MEMS Sensor/Actuator/Locator

9. Needed Expedition Expertise
MEMS and hardware devices
Scalable computing architectures
Networked-oriented operating systems
Distributed file systems
Data management systems
Security/privacy
User interfaces
Collaboration applications
Intelligent learning systems
Program verification
Methodologies for HW/SW design/evaluation

10. Interdisciplinary, Technology-Centered Expedition Team
Alex Aiken, PL
Eric Brewer, OS
John Canny, AI
David Culler, OS/Arch
Joseph Hellerstein, DB
Michael Jordan, Learning
Anthony Joseph, OS
Randy Katz, Nets
John Kubiatowicz, Arch
James Landay, UI
Jitendra Malik, Vision
George Necula, PL
Christos Papadimitriou, Theory
David Patterson, Arch
Kris Pister, Mems
Larry Rowe, MM
Alberto Sangiovanni-Vincentelli, CAD
Doug Tygar, Security
Robert Wilensky, DL/AI

11. Organization: The Expedition Cube
Information Devices
Information Utility
Applications D e s I g n M t h o d l y
MEMS Sensors/Actuators, Smart Dust,
Radio Tags, Cameras, Displays,
Communicators, PDAs
Fluid Software, Cooperating Components,
Diverse Device Support, Sensor-Centric
Data Mgmt, Always Available, Tacit
Information Exploitation (event modeling)
Rapid Decision Making, Learning,
Smart Spaces: Collaboration Rooms,
Classrooms, Vehicles
Base Program
Option 1: Sys Arch for Diverse Devices
Option 2: Oceanic Data Utility
Option 4: Negotiation Arch for Cooperation
Option 5: Tacit Knowledge Infrastructure
Option 6: Classroom Testbed
Option 7: Scalable Heterogeneous Component-Based Design
Option 3: Capture and Re-Use

12. Base Program: Leader Katz
Broad but necessarily shallow investigation into all technologies/applications of interest
Primary focus on Information Utility
No new HW design: commercially available information devices
Only small-scale testbed in Soda Hall
Fundamental enabling technologies for Fluid Software
Partitioning and management of state between soft and persistent state
Data and processing placement and movement
Component discovery and negotiation
Flexible capture, self-organization, info re-use
Limited Applications
Methodology: Formal Methods & User-Centered Design

13. Option 1: “System Architecture for Vastly Diverse Devices” Leader Culler
Distributed control & resource management: data mvmt & transformation, not processing
Path concept for information flow, not the thread
Persistent state in the infrastructure, soft state in the device
Non-blocking system state, no application state in the kernel
Functionality not in device is accessible thru non-blocking remote method invocation
Extend the Ninja concepts (thin client/fat infrastructure) beyond PDAs to MEMS devices, cameras, displays, etc.

14. Option 2: Implementation & Deploy-ment of Oceanic Data Info Utility Leader Kubiatowicz
Nomadic Data Access: serverless, homeless, freely flowing thru infrastructure
Opportunistic data distribution
Support for: promiscuous caching; freedom from administrative boundaries; high availability and disaster recovery; application-specific data consistency; security
Data Location and Consistency
Overlapping, partially consistent indices
Data freedom of movement
Expanding search parties to find data, using application-specific hints (e.g., tacit information)

15. Option 3: Sensor-Centric Data Management for Capture/Reuse Leader Hellerstein
Integration of embedded MEMS with software that can extract, manage, analyze streams of sensor-generated data
Wide-area distributed path-based processing and storage
Data reduction strategies for filtering/aggregation
Distributed collection and processing
New information management techniques
Managing infinite length strings
Application-specific filtering and aggregation
Optimizing for running results rather than final answers
Beyond data mining to “evidence accumulation” from inherently noisy sensors

16. Option 4: Negotiation Architecture for Cooperating Components Leader Wilensky
Self-administration through auto-discovery and configuration among confederated components
Less brittle/more adaptive systems
Negotiation Architecture
Components announce their needs and services
Service discovery and rendezvous mechanisms to initiate confederations
Negotiated/contractural APIs: contract designing agents
Compliance monitoring and renegotiation
Graceful degradation in response to environmental changes

17. Option 5: Tacit Knowledge Infra-structure/Rapid Decision Making Leader Canny
Exploit information about the flow of information to improve collaborative work
Capture, organize, and place tacit information for most effective use
Learning techniques: infer communications flow, indirect relationships, and availability/participation to enhance awareness and support opportunistic decision making
New collaborative applications
3D “activity spaces” for representing decision-making activities, people, & information sources
Visual cues to denote strength of ties between agents, awareness levels, activity tracking, & attention span

18. Option 6: Info Mgmt for Intelligent Classrooms Leader Joseph
Electronic Problem-based Learning
Collaborative learning enabled by information appliances
Enhanced Physical and Virtual Learning Spaces
Wide-area, large-scale group collaboration
Capture interaction once for replay
Preference/task-driven information device selection
Service accessibility
Device connectivity
Wide-area support
Iterative evaluation

19. Option 7: Safe Component Design and UI Design Tools Leader Sangiovanni
Information Appliances as an application of hardware/software codesign
Co-design Finite State Machines (CFSMs)
Formal methods to verify safety from faults
Safe partitioning of components into communicating subcomponents placed into the wide-area
Model-based User Interface Tools
Information device user interfaces
Multimodal interface design for variety of devices

20. Option 8: Scaled-up Field Trials Leader Katz
Testbed Rationale
Study impact on larger/more diverse user community
Higher usage levels to stress underlying architecture
Make commitment to true utility functionality
Increasing Scale of Testbeds
Building-Scale
Order 100s individuals
Campus-Scale
Order 1000s individuals
City-Scale
Order individuals

21. Putting It All Together
1. Diverse Devices
2. Data Utility
3. Capture/Reuse
4. Negotiation
5. Tacit Knowledge
6. Classroom
7. Design Methods
8. Scale-up
Devices
Utility
Applications
Component Discovery
& Negotiation
Fluid Software
Info Extract/Re-use
Self-Organization
Group Decision Making
Learning

22. Letters of Support
AT&T Labs, Research: Dr. Hamid Ahmadi, Networking and Distributed Systems Research Vice President
Cadence: Dr. Patrick Scaglia, VP Research, Cadence Laboratories
Hewlett Packard: Dr. Steve Rosenberg, Manager, External Research, HP Labs
IBM: Dr. William Cody, Manager, Exploratory Database Systems
Intel: Dr. Richard Wirt, Director, Intel Microcomputer Laboratory
Lucent/Bell Labs: Dr. William M. Coughran, Jr., Bell Labs Research Silicon Valley Vice President

23. Letters of Support
Microsoft: Dr. Daniel Ling, Director, Microsoft Research
Motorola: Dr. John Barr, Director, System of Systems Architecture, Personal Information Networking Division
Nortel Networks: Dr. Daniel Pitt, VP Technology and Director Bay Architecture Lab
Sprint: Dr. Frank Denap, Director, Advanced Technology Labs
Sun Microsystems: Dr. Greg Papadopoulos, Vice President and Chief Technology Officer
Xerox: Dr. Mark Weiser, Chief Technologist, Palo Alto Research Center

24. Letters of Support

25. Discussion