Unique Opportunities in Experimental Computer Systems Research - the Berkeley Testbeds David Culler U.C. Berkeley Grad Student Presentations 8/27/1999

8/26/99Grad Presentation2 Emerging Problems of Scale Scalable, Available Internet Services Info. appliances Client Server Clusters Massive Cluster Gigabit Ethernet

8/26/99Grad Presentation3 Convergence at the Extremes Powerful Services on “Small” Devices –massive computing and storage in the infrastructure –active adaptation of form and content “along the way” Extremes more alike that either is to the middle –More specialized in function –Communication centric design »wide range of networking options –Federated System of Many Many Systems –Hands-off operation, mgmt, development –High Reliability, Availability –Scalability –Power and space limited –simplicity They have to “work or die!”

8/26/99Grad Presentation4 100 node Ultra/Myrinet NOW GLUnix Active Messages xFS Fast sockets, MPI, and SVM Titanium and Split-C ScaLapack

8/26/99Grad Presentation5 Novel Systems Design Virtual networks –integrate communication events into virtual memory system Implicit Co-scheduling –cause local schedulers to co-schedule parallel computations using a two-phase spin-block and observing round-trip Co-operative caching –access remote caches, rather than local disk, and enlarge global cache coverage by simple cooperation Reactive Scalable I/O Network virtual memory, fast sockets ISAAC “active” security Internet Server Architecture TACC Proxy architecture

8/26/99Grad Presentation6 The Millennium Vision To work, think, and study in a computationally rich environment with deep information stores and powerful services –test ideas through simulation –explore and investigate data and information –share, manipulate, and interact through natural actions Organized in a manner consistent with the University setting –clusters of clusters –Computational Economy Novel modes of interacting with large amounts of data

8/26/99Grad Presentation7 The Millennium Community School of Info. Mgmt and Sys. Computer Science Electrical Eng. Mechanical Eng. BMRC Nuclear Eng. IEOR Civil Eng. MSME Inst. Of Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation8 NT Workstations for Sci. & Eng. SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation9 SMP => storage, small-scale parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation10 Group Cluster of SMPs => Parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME NERSC Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation11 Campus Cluster => large-scale Parallelism SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME NERSC Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation12 Gigabit Ethernet Connectivity Gigabit Ethernet SIMS C.S. E.E. M.E. BMRC N.E. IEOR C. E. MSME NERSC Transport Business Chemistry Astro Physics Biology Economy Math

8/26/99Grad Presentation13 FIAT LUX: crossing areas Combines –Image Based Modeling and Rendering, –Image Based Lighting, –Dynamics Simulation and –Global Illumination in a completely novel fashion to achieve unprecedented levels of scientific accuracy and realism Computing Requirements –15 Days of worth of time for development. –5 Days for rendering Final piece. –4 Days for rendering in HDTV resolution on 140 Processors Storage –72,000 Frames, 108 Gigabytes of storage –7.2 Gigs after motion blur –500 MB JPEG premiere at the SIGGRAPH 99 Electronic Theater –

8/26/99Grad Presentation14 An upcoming Comp. Econ Experiment Two identical 32 proc Millennium Clusters One open shop One with usage based on bid-based proportional share scheduling

8/26/99Grad Presentation15 Ninja: Push Services into an Active Infrastr. Servers Clients Servers Infrastructure Services Open => enable Distributed Innovation of Scalable, Avail. Services

8/26/99Grad Presentation16 Universal Computing Lab (464 Soda) Computing in the infra, in the walls, on the desk, in your hand,... Not just a new project, a new computing culture

8/26/99Grad Presentation17 universal Function: adjective 1 : including or covering all or a whole collectively or distributively without limit or exception 2 a : present or occurring everywhere b : existent or operative everywhere or under all conditions 3 a : embracing a major part or the greatest portion (as of mankind) b : comprehensively broad and versatile 4 a : affirming or denying something of all members of a class or of all values of a variable b : denoting every member of a class 5 : adapted or adjustable to meet varied requirements (as of use, shape, or size)