Presentation is loading. Please wait.

Presentation is loading. Please wait.

Next-Generation Internet-Scale Systems Ninja/Millennium Projects

Published byMaria Eduarda Varejão Faria Modified over 6 years ago

Similar presentations

Presentation on theme: "Next-Generation Internet-Scale Systems Ninja/Millennium Projects"— Presentation transcript:

1 Next-Generation Internet-Scale Systems Ninja/Millennium Projects
David E. Culler Computer Science Division U.C. Berkeley ninja.cs.berkeley.edu

2 Future Internet-Scale Systems
Information Devices (~10 Billion) Connected Stationary Computers (~100 Million) Scalable Servers (~Million) 12/5/2018 Ninja Overview

3 Internet-Scale Systems
Extremely large, complex, distributed, heterogeneous, with continuous and rapid introduction of new technologies Feasible architectures Decentralized, scalable algorithms Dynamically deployed agents where they are needed; “Big infrastructure, small clients” Incremental processing/communications growth Careful violation of traditional layering Implementation approach based on incremental prototyping, deployment, evaluation, experimentation 12/5/2018 Ninja Overview

4 Imagine You walk into a room
Your PDA connects to the local infrastructure and asks it to build a custom GUI You walk into a room Next, your PDA asks the infrastructure for a path out to your personal information space, where agents are processing your , v-mail, faxes, and pages You have complete, secure, optimized access to local devices and your private resources 12/5/2018 Ninja Overview

5 Vision  Goal The next internet revolution will come from enabling component services and pervasive access not vertically integrated Information Technology  Dynamic, programmatic creation / composition of scalable, highly available, & customizable services Automatic adaptation to end device characteristics and network connectivity Arbitrarily powerful services on arbitrarily small clients using a proactive infrastructure 12/5/2018 Ninja Overview

6 NOW Project Breakthrough design techniques for large-scale systems
Low-overhead interface to enable emerging networks Active Messages Explore novel OS designs World-leading cluster Basis for Inktomi 12/5/2018 Ninja Overview

7 Spoon feed web pages to PDAs
Transcoding Proxies Laptops, Desktops Info. Appliances Network Computers Spoon feed web pages to PDAs Scalable Servers (NOW) Legacy Servers Transformation, Aggregation, Caching, and Customization (TACC) Scalability and availability Limited customizability and locality and no persistence 12/5/2018 Ninja Overview

8 ProActive Approach Create a framework that enables programmatic generation and composition of services out of strongly typed reusable components Key Elements Structured architecture with a careful partitioning of state Bases, Active Routers, and Units Wide-area paths formed out of strongly-typed components Operators and Connectors Execution environments with efficient, but powerful communication primitives Active Messages + capsules TACC + persistence + customization 12/5/2018 Ninja Overview

9 Structured Architecture
Bases highly available persistent state (safe) databases, agents “home” base per user service programming environment Wide-Area Path Active Proxies not packet routers soft-state well-connected localization (any to any) Units sensors / actuators PDAs / smartphones / PCs heterogeneous Minimal functionality: “Smart Clients” 12/5/2018 Ninja Overview

10 Example: TopGun WingMan/Mediaboard Operation
AP PDA Legacy Server PDA Proxy Base Image Converter Un-Zip MediaBoard PC Multicast Connector Aggregator AP Mic Camera 12/5/2018 Ninja Overview

11 NINJA Service Architecture
Operators/Connectors/Interfaces Paths Wide-Area Paths Interface Interconnection Path Optimization Units, Active Proxies, and Bases Services Service Discovery Automatic Path Generation Example Applications Universal remote control/smart spaces Universal In-Box/Personal Information Management 12/5/2018 Ninja Overview

12 Operators/Connectors
transformation aggregation agents PI provides secure execution environment Connectors: abstract wires ADUs varying semantics uni/multicast Interfaces: strongly typed language independent control channel path changes authentication feedback 12/5/2018 Ninja Overview

13 Wide-Area Paths Creation (explicit or automatic): Optimization:
Query Service Discovery Service to find logical path of operators Place operators onto nodes: Path is unit of resource allocation and authentication Connectors are polymorphic: entire path must type check - statically Optimization: Add (or transpose) operators forward error-correction compression/decompression Change operators, connectors, locations, or parameters Interoperability: Wrapper operators for legacy servers Leverage COM objects as operators 12/5/2018 Ninja Overview

14 iSpace Execution Environment
operator upload Service request service threads Persistent Storage Managed RMI++ Physical processor Operators Caches parallel application framework on Bases RMI++ hides complexity of scalability and availability Dynamic customization and composition apSpace is limited execution environment for AR 12/5/2018 Ninja Overview

15 Base Execution Environment
Ninja RMI Sun RMI compatible serialization and thread management ninja remote object + TCP or UDP or Multicast UDP (Active Msg soon) + Authenticated public key iS-box customizable service VM Redirector = iSpace 12/5/2018 Ninja Overview

16 iS-box: platform framework
Loader Extends JVM to support services LoadService (URL, name, args) ListServices GetService(name) -> svc obj KillService Trusted services loaded at startup Security MGR interposes on method calls loaded as a trusted service iS-Loader Trusted-Services Security MGR JVM 12/5/2018 Ninja Overview

17 Push Services into the Infrastructure
New service iS-Loader Trusted-Services Service Methods RMI stubs Security MGR JVM Generated by RMI compiler GetService returns service object Issue: Programming Model for Service Methods 12/5/2018 Ninja Overview

18 Scalable iSpace Multi-Space services across group of iS-boxes
System Area Network Node iS-box Multi-Space Multi-Space services across group of iS-boxes List, Get, or Load Service from any Get returns redirector stub iS-Loader Multi-Space Loader Multi-Space SVC Security MGR JVM 12/5/2018 Ninja Overview

19 Prototype Services iSpace Admin monitor automatic forms interface
computational econ support juke box ninja fax pilot pager Digicash mint 1200 RMI/s on 400 PII w/ jit 12/5/2018 Ninja Overview

20 Millennium Project Goals
Enable major advances in Computational Science and Engineering Simulation, Modeling, and Information Processing becoming ubiquitous Explore novel design techniques for large, complex systems Fundamental Computer Science problems ahead are problems of scale Develop fundamentally better ways of assimilating and interacting with large volumes of information and with each other Explore emerging technologies networking, OS, devices 12/5/2018 Ninja Overview

21 Campus-wide Millennium Testbed
Desktop PCs Servers Clusters Massive Cluster Gigabit Ethernet PDAs Cell Phones Future Devices Wireless Infrastructure 12/5/2018 Ninja Overview

22 Physical Connectivity
12/5/2018 Ninja Overview

23 Resource Support Computers via Intel Technology 2000 grant
200 NT desktops 16 department 4-way SMPs 8 5x4 Group Clusters, 1 ~100x4 Campus Cluster PPro => Pentium II => Merced Additional storage via IBM SUR grant 0.5 TB this year => 4 TB NT tools via Microsoft grant Solaris x86 tools via SMCC grant Campus provides Technical staff NorTel Discount NSF Research Infrastructure Grant 200 Gflop/s 150 GB memory 8 TB disk 12/5/2018 Ninja Overview

24 High Speed Cluster Networking
Transforms large collection of individual resources into a powerful system Emerging system area networks Myrinet, ServerNet, Synfinity Emerging user-level networking Virtual Interface Architecture Intel/Microsoft/Compaq std based on Active Messages and related research NT and Unix Gigabit Ethernet? 12/5/2018 Ninja Overview

25 Inter-cluster NorTel network
Gigabit Ethernet connecting group clusters and campus cluster Campus provides fiber plant, maintenance, and staff 12/5/2018 Ninja Overview

26 Computational Economy
How is this vast, integrated pool of resources managed? Traditional system approach: empower global OS to provide “optimal” allocation to blind applications predefined metric, tuned to fixed workload ignores the inherent adaptation of demand Computer Center charge => director-to-user feedback according to cost Economic view: decentralized allocation according to perceived value pricing => user-to-user feedback compatible niches,sense of control, cooperation Everything is a service! 12/5/2018 Ninja Overview

27 Experiment: PDA Bazaar
Deploy/use pervasive computing infrastructure in Soda Hall Provide Ninja iSpaces Build an initial community (200 PP3 / Workpads) Watch and evaluate information broadcast channels seminars, lecture content news/sports/stocks shared information calendars, room reservations collaborative note-taking and brainstornming Smart spaces and device control 12/5/2018 Ninja Overview

28 Perspective Computer Science Research Challenges Ahead are fundamentally problems of scale and composition Powerful services Billions of tiny devices Rapid development, deployment, and evolution These demand an entirely new approach to systems research. Academic / Industry collaboration is key 12/5/2018 Ninja Overview

Download ppt "Next-Generation Internet-Scale Systems Ninja/Millennium Projects"

Similar presentations

System Area Network Abhiram Shandilya 12/06/01. Overview Introduction to System Area Networks SAN Design and Examples SAN Applications.

System Area Network Abhiram Shandilya 12/06/01. Overview Introduction to System Area Networks SAN Design and Examples SAN Applications.
NDN in Local Area Networks Junxiao Shi The University of Arizona 2014-09-04 1.

NDN in Local Area Networks Junxiao Shi The University of Arizona
Unique Opportunities in Experimental Computer Systems Research - the Berkeley Testbeds David Culler U.C. Berkeley Grad.

Unique Opportunities in Experimental Computer Systems Research - the Berkeley Testbeds David Culler U.C. Berkeley Grad.
1 Internet-Scale Systems Research Group (ISRG) Eric Brewer, David Culler, Anthony Joseph, Randy Katz Computer Science Division, EECS Department University.

1 Internet-Scale Systems Research Group (ISRG) Eric Brewer, David Culler, Anthony Joseph, Randy Katz Computer Science Division, EECS Department University.
NPACI Panel on Clusters David E. Culler Computer Science Division University of California, Berkeley

NPACI Panel on Clusters David E. Culler Computer Science Division University of California, Berkeley
Ninja and the Post-PC Era David Culler U.C. Berkeley Mar 12, 1999

Ninja and the Post-PC Era David Culler U.C. Berkeley Mar 12, 1999
Millennium Overview and Status David Culler and Jim Demmel Computer Science Division

Millennium Overview and Status David Culler and Jim Demmel Computer Science Division
Towards I-Space Ninja Mini-Retreat June 11, 1997 David Culler, Steve Gribble, Mark Stemm, Matt Welsh Computer Science Division U.C. Berkeley.

Towards I-Space Ninja Mini-Retreat June 11, 1997 David Culler, Steve Gribble, Mark Stemm, Matt Welsh Computer Science Division U.C. Berkeley.
1 Internet-Scale Systems Research Group Eric Brewer, David Culler, Anthony Joseph, Randy Katz, Steven McCanne Computer Science Division, EECS Department.

1 Internet-Scale Systems Research Group Eric Brewer, David Culler, Anthony Joseph, Randy Katz, Steven McCanne Computer Science Division, EECS Department.
EJB Design. Server-side components Perform –complex algorithms –high volume transactions Run in –highly available environment (365 days/year) –fault tolerant.

EJB Design. Server-side components Perform –complex algorithms –high volume transactions Run in –highly available environment (365 days/year) –fault tolerant.
EECS Research into the Post-PC Era David Culler U.C. Berkeley Feb 25, 1999

EECS Research into the Post-PC Era David Culler U.C. Berkeley Feb 25, 1999
Finale’ cs294-8 Design of Deeply Networked Systems Spring 2000 David Culler & Randy Katz U.C. Berkeley

Finale’ cs294-8 Design of Deeply Networked Systems Spring 2000 David Culler & Randy Katz U.C. Berkeley
ProActive Infrastructure Eric Brewer, David Culler, Anthony Joseph, Randy Katz Computer Science Division U.C. Berkeley ninja.cs.berkeley.edu Active Networks.

ProActive Infrastructure Eric Brewer, David Culler, Anthony Joseph, Randy Katz Computer Science Division U.C. Berkeley ninja.cs.berkeley.edu Active Networks.
Brewer’s Endeavor Goals Make the fluid infrastructure an extension of the Ninja services frameworkMake the fluid infrastructure an extension of the Ninja.

Brewer’s Endeavor Goals Make the fluid infrastructure an extension of the Ninja services frameworkMake the fluid infrastructure an extension of the Ninja.
Big Infrastructure, Small Clients Prof. Eric A. Brewer

Big Infrastructure, Small Clients Prof. Eric A. Brewer
Connecting the Invisible Extremes of Computing David Culler U.C. Berkeley Summer Inst. on Invisible Computing July,

Connecting the Invisible Extremes of Computing David Culler U.C. Berkeley Summer Inst. on Invisible Computing July,
Chapter 7: Client/Server Computing Business Data Communications, 5e.

Chapter 7: Client/Server Computing Business Data Communications, 5e.
Clusters Massive Cluster Gigabit Ethernet System Design for Vastly Diverse Devices David Culler U.C. Berkeley HP Visit 3/9/2000.

Clusters Massive Cluster Gigabit Ethernet System Design for Vastly Diverse Devices David Culler U.C. Berkeley HP Visit 3/9/2000.
1 Internet-Scale Systems Research Group Eric Brewer, David Culler, Anthony Joseph, Randy Katz, Steven McCanne Computer Science Division, EECS Department.

1 Internet-Scale Systems Research Group Eric Brewer, David Culler, Anthony Joseph, Randy Katz, Steven McCanne Computer Science Division, EECS Department.
Post-PC Summary Prof. Eric A. Brewer

Post-PC Summary Prof. Eric A. Brewer

Similar presentations

Ads by Google