1 The Endeavour Expedition: Charting the Fluid Information Utility Randy H. Katz, Principal Investigator EECS Department University of California, Berkeley Berkeley, CA

2 Why “Endeavour”? DARPA BAA 99-07: Information Technology Expeditions To strive or reach; a serious determined effort (Webster’s 7th New Collegiate Dictionary); British spelling Captain Cook’s ship from his first voyage of exploration of the great unknown of his day: the southern Pacific Ocean ( ). –These voyages brought brought more land and wealth to the British Empire than any military campaign. –Cook’s lasting contribution: comprehensive knowledge of the people, customs, and ideas that lay across the sea –“He left nothing to his successors other than to marvel at the completeness of his work.”

3 Expedition Goals Enhancing human understanding through information technology –Dramatically more convenient for people to interact with information, devices, and other people –Supported by a “planetary-scale” Information Utility »Stress tested by challenging applications in decision making and learning »New methodologies for design, construction, and administration of systems of unprecedented scale and complexity –Figure of merit: how effectively we amplify and leverage human intellect A pervasive Information Utility, based on “fluid systems technology” to enable new approaches for problem solving & learning

4 Expedition Assumptions Human time and attention, not processing or storage, are the limiting factors Givens: –Vast diversity of computing devices (PDAs, cameras, displays, sensors, actuators, mobile robots, vehicles); No such thing as an “average” device –Unlimited storage: everything that can be captured, digitized, and stored, will be –Every computing device is connected in proportion to its capacity –Devices are predominately compatible rather than incompatible (plug-and-play enabled by on-the-fly translation/adaptation)

5 Expedition Challenges Personal Information Mgmt is the Killer App –Not corporate processing but management, analysis, aggregation, dissemination, filtering for the individual People Create Knowledge, not Data –Not management/retrieval of explicitly entered information, but automated extraction and organization of daily activities Information Technology as a Utility –Continuous service delivery, on a planetary-scale, on top of a highly dynamic information base Beyond the Desktop –Community computing: infer relationships among information, delegate control, establish authority

6 Expedition Approach Information Devices –Beyond extrapolated desktop devices to MEMS- sensors/actuators plus capture/display to yield enhanced activity spaces Information Utility –“Fluid”, Network-Centric System Software »Paths/Streams: process/store/manage information »“Movable” Processing and Storage »Partitioned/distributed functionality Thin-Clients/Fat-Infrastructure »Nomadic Data »Negotiation-based Interfaces »Always-Available Functionality –Wide-area distributed coordination and control on scalable servers

7 Expedition Approach Information Applications –High Speed/Collaborative Decision Making, Learning –Augmented “Smart” Spaces: Rooms, Vehicles Design Methodology –HW/SW Co-design –Formal Methods –Decomposable and Reusable Components –User-centered Design

8 Information Utility Information Devices Applications Collaboration Spaces High Speed Decision Making Learning Classroom Info Appliances E-BookVehicles PDA Handset Laptop Camera Smartboard MEMS Sensor/Actuator/Locator Wallmount Display Generalized UI Support Proxy Agents Human Activity Capture Event Modeling Transcoding, Filtering, Aggregating Statistical Processing/Inference Negotiated APIsSelf-Organizing Data Interface ContractsWide-area Search & Index Nomadic Data & Processing Automated Duplication Distributed Cache Management Wide-Area Data & Processing Movement & Positioning Stream- and Path-Oriented Processing & Data Mgmt Non-Blocking RMISoft-/Hard-State Partitioning

9 Needed Expedition Expertise MEMS and hardware devices Scalable computing architectures Networked-oriented operating systems Distributed file systems Data management systems Security/privacy User interfaces Collaboration applications Intelligent learning systems Program verification Methodologies for HW/SW design/evaluation

10 Interdisciplinary, Technology- Centered Expedition Team Alex Aiken, PL Eric Brewer, OS John Canny, AI David Culler, OS/Arch Joseph Hellerstein, DB Michael Jordan, Learning Anthony Joseph, OS Randy Katz, Nets John Kubiatowicz, Arch James Landay, UI Jitendra Malik, Vision George Necula, PL Christos Papadimitriou, Theory David Patterson, Arch Kris Pister, Mems Larry Rowe, MM Alberto Sangiovanni- Vincentelli, CAD Doug Tygar, Security Robert Wilensky, DL/AI

11 Organization: The Expedition Cube Information Devices Information Utility Applications DesIgnDesIgn MethodologyMethodology MEMS Sensors/Actuators, Smart Dust, Radio Tags, Cameras, Displays, Communicators, PDAs Fluid Software, Cooperating Components, Diverse Device Support, Sensor-Centric Data Mgmt, Always Available, Tacit Information Exploitation (event modeling) Rapid Decision Making, Learning, Smart Spaces: Collaboration Rooms, Classrooms, Vehicles Base Program Option 1: Sys Arch for Diverse Devices Option 2: Oceanic Data Utility Option 4: Negotiation Arch for Cooperation Option 5: Tacit Knowledge Infrastructure Option 6: Classroom Testbed Option 7: Scalable Heterogeneous Component-Based Design Option 3: Capture and Re-Use

12 Base Program: Leader Katz Broad but necessarily shallow investigation into all technologies/applications of interest –Primary focus on Information Utility »No new HW design: commercially available information devices »Only small-scale testbed in Soda Hall –Fundamental enabling technologies for Fluid Software »Partitioning and management of state between soft and persistent state »Data and processing placement and movement »Component discovery and negotiation »Flexible capture, self-organization, info re-use –Limited Applications –Methodology: Formal Methods & User-Centered Design

13 Option 1: “System Architecture for Vastly Diverse Devices” Leader Culler Distributed control & resource management: data mvmt & transformation, not processing –Path concept for information flow, not the thread –Persistent state in the infrastructure, soft state in the device –Non-blocking system state, no application state in the kernel –Functionality not in device is accessible thru non-blocking remote method invocation Extend the Ninja concepts (thin client/fat infrastructure) beyond PDAs to MEMS devices, cameras, displays, etc.

14 Option 2: Implementation & Deploy- ment of Oceanic Data Info Utility Leader Kubiatowicz Nomadic Data Access: serverless, homeless, freely flowing thru infrastructure –Opportunistic data distribution –Support for: promiscuous caching; freedom from administrative boundaries; high availability and disaster recovery; application-specific data consistency; security Data Location and Consistency –Overlapping, partially consistent indices –Data freedom of movement –Expanding search parties to find data, using application- specific hints (e.g., tacit information)

15 Option 3: Sensor-Centric Data Management for Capture/Reuse Leader Hellerstein Integration of embedded MEMS with software that can extract, manage, analyze streams of sensor-generated data –Wide-area distributed path-based processing and storage –Data reduction strategies for filtering/aggregation –Distributed collection and processing New information management techniques –Managing infinite length strings –Application-specific filtering and aggregation –Optimizing for running results rather than final answers –Beyond data mining to “evidence accumulation” from inherently noisy sensors

16 Option 4: Negotiation Architecture for Cooperating Components Leader Wilensky Cooperating Components –Self-administration through auto-discovery and configuration among confederated components –Less brittle/more adaptive systems Negotiation Architecture –Components announce their needs and services –Service discovery and rendezvous mechanisms to initiate confederations –Negotiated/contractural APIs: contract designing agents –Compliance monitoring and renegotiation –Graceful degradation in response to environmental changes

17 Option 5: Tacit Knowledge Infra- structure/Rapid Decision Making Leader Canny Exploit information about the flow of information to improve collaborative work –Capture, organize, and place tacit information for most effective use –Learning techniques: infer communications flow, indirect relationships, and availability/participation to enhance awareness and support opportunistic decision making New collaborative applications –3D “activity spaces” for representing decision-making activities, people, & information sources –Visual cues to denote strength of ties between agents, awareness levels, activity tracking, & attention span

18 Option 6: Info Mgmt for Intelligent Classrooms Leader Joseph Electronic Problem-based Learning –Collaborative learning enabled by information appliances Enhanced Physical and Virtual Learning Spaces –Wide-area, large-scale group collaboration –Capture interaction once for replay –Preference/task-driven information device selection –Service accessibility –Device connectivity –Wide-area support –Iterative evaluation

19 Option 7: Safe Component Design and UI Design Tools Leader Sangiovanni Information Appliances as an application of hardware/software codesign –Co-design Finite State Machines (CFSMs) –Formal methods to verify safety from faults –Safe partitioning of components into communicating subcomponents placed into the wide-area Model-based User Interface Tools –Information device user interfaces –Multimodal interface design for variety of devices

20 Option 8: Scaled-up Field Trials Leader Katz Testbed Rationale –Study impact on larger/more diverse user community –Higher usage levels to stress underlying architecture –Make commitment to true utility functionality Increasing Scale of Testbeds –Building-Scale »Order 100s individuals –Campus-Scale »Order 1000s individuals –City-Scale »Order individuals

21 Putting It All Together 1. Diverse Devices 2. Data Utility 3. Capture/Reuse 4. Negotiation 5. Tacit Knowledge 6. Classroom 7. Design Methods 8. Scale-up Devices Utility Applications Fluid Software Info Extract/Re-use Group Decision Making Learning Component Discovery & Negotiation Self-Organization

22 Letters of Support AT&T Labs, Research: Dr. Hamid Ahmadi, Networking and Distributed Systems Research Vice President Cadence: Dr. Patrick Scaglia, VP Research, Cadence Laboratories Hewlett Packard: Dr. Steve Rosenberg, Manager, External Research, HP Labs IBM: Dr. William Cody, Manager, Exploratory Database Systems Intel: Dr. Richard Wirt, Director, Intel Microcomputer Laboratory Lucent/Bell Labs: Dr. William M. Coughran, Jr., Bell Labs Research Silicon Valley Vice President

23 Letters of Support Microsoft: Dr. Daniel Ling, Director, Microsoft Research Motorola: Dr. John Barr, Director, System of Systems Architecture, Personal Information Networking Division Nortel Networks: Dr. Daniel Pitt, VP Technology and Director Bay Architecture Lab Sprint: Dr. Frank Denap, Director, Advanced Technology Labs Sun Microsystems: Dr. Greg Papadopoulos, Vice President and Chief Technology Officer Xerox: Dr. Mark Weiser, Chief Technologist, Palo Alto Research Center

24 Letters of Support

25 Discussion