Clusters Massive Cluster Gigabit Ethernet Progress on System Architecture for Extreme Devices David Culler U.C. Berkeley Endeavour Retreat 1/20/200

1/20/2000Endeavour Sys. Arch2 Outline The Very Large –Millenium Cluster-of-cluster resources available to you –System Architecture The Middle –Kiosks, laptops, and PDAs The Small –Embedded Servers –Low power wireless –Architecture for Zillions of devices

1/20/2000Endeavour Sys. Arch3 Large Resource Deployment In place: –Full 64-PIII Linux Cluster w/ Myrinet & ethernet (Gb rdy) »demonstrated on Ninja DDS –several remote 16-PIII clusters –DLIB 4-P + 1/4 TB cluster –1/2 TB storage server –Gb connection to dept, OC48 to NTON –Rootstock cluster dissemination facility –REXEC econ-based remote execution facility –100 KVA PDU Demonstrated –intercampus network, routing, config (as per CNS) –full Gb/s with 2 source, 2 sinks Deploying –45 x 4 of main cluster (2/4 GB mem, 18 GB disk, Gb ether)

1/20/2000Endeavour Sys. Arch4 Vineyard Cluster Architecture dissemination, resource control, remote execution, and communication - VIA / GM, GbE - Multicast Applications / Services (ISPACE/Kiosks) - NT / Linux (2.2.x) - Stride Scheduler MPIVEXEC PBS I/O Mgmt / Monitoring REXEC TOOLS Rootstock Distribution

1/20/2000Endeavour Sys. Arch5 Dissemination: Rootstock K cluster stock - build - os - drvrs - mill SW - os mods leased builds cs CAN Cluster System Distribution Center... IP network 1. Cluster Stock - Rootstock build pages - Full Current Linux - all fixes and pckgs - SSL, SSH - Cluster Drivers - Cluster System Layers - rexec, mpe, pbs - Optional SW ($) - Cluster Kernal Mods 5. Cluster Update button (future) - 2nd dialtone, CF engine, rolling update 2. Make the CS “graft” - specify IP address - pckg removes - dchp, dns, nis,... sanity check and build - resolv.conf, /etc/hosts,... constructs cluster build (lease) download CS build floppy Cluster 3. CS power-on build - xfer and localize DT - add local admin scripts - node build floppy 4. Node power-on build - local stock from CS

1/20/2000Endeavour Sys. Arch6 REXEC / VEXEC Resource Management, Autoconfig, Mechanism/Policy, Enforcement rexecd vexecd (Policy A) rexec Cluster IP Multicast Channel %rexec –n 2 –r 3 indexer minimum $ vexecd (Policy B) Node ANode BNode CNode D “Nodes AB” run indexer on Nodes AB at 3 credits/min

1/20/2000Endeavour Sys. Arch7 “Intelligent” Middle Deployed many laptops with , 3 base stations, many PDAs with IR Solved the PDA to IR-dongle and PDA to Annex serial port (J. Hill) Deployed two kiosks: touch-LCD, IR-ppp –act as server for managment Demonstrated key aspects of the service infrastructure –eg: get device applet from service point eSticky notes appln motivated xcoding-security infrastucture => Need to harvest and extend

1/20/2000Endeavour Sys. Arch8 Small: Embedded Servers Tested commercial products - promising –axis camera server –SOHO NAT, DHCP, Firewall server plumbing = ethernet Identified platform for building embedded servers –DIMM PC + Linux + … – MB RAM + 16 MB Flash Disk

1/20/2000Endeavour Sys. Arch9 Low-power Wireless Tested available options –RF monolithics (used in Smart Rocks) »“virtual wire” is brain-dead, but good tranceiver –RadioMetrics (used in ISI RF Tags) »simple, primitive packet controller, no pwr down –World Wireless »nice MAC, but only infrastructure mode Selected RF monolithics –working with BSAC and ISI on building-block –new packet controller + MAC

1/20/2000Endeavour Sys. Arch10 Zillions of Little Devices Connected device as client well-established –distiller in the infrastructure spoonfeeds client »powerful services in power-limited devices! –How to get the illusion of continuous connectivity? What about sensors-based devices? –they should behave as servers »eg: camera server –How to scale tiny server to need? –How to get illusion of continuous connectivity? »use the infrastructure First a demonstration: note server in a PDA

1/20/2000Endeavour Sys. Arch11 Assumptions Computation and storage in the infrastructure is plentiful Wired bandwidth is pervasive and essentially free [ Multicast is widely accessible] => every device has a representative proxy in the infrastructure

1/20/2000Endeavour Sys. Arch12 Cautionary enabling thought TCP protocol (acks, fsm, etc.) terminates in the kernel. System specific protocol exists between kernel TCP agent and applications. –SSI clusters have another layer of network! Think of device as an application (not big laptop) Where is its kernel agent? IP TCP app

1/20/2000Endeavour Sys. Arch13 Device Access Architecture infra proxy provides name, state, queuing, etc. extend toward AP as optimization Physical Device low power local device link AP Scalable, Available Ninja Base Clients Services persistent named representative Dev MC

1/20/2000Endeavour Sys. Arch14 Demo Mapping PDA serial AP Laptop w/ Ninja iSpace Laptop Browser persistent named representative TCP laptop w/ pppd BayStacker Wireless

1/20/2000Endeavour Sys. Arch15 Key piece to build Low power controller with 2 stream devices –X = sensor + actuator for devices –X = host interface for AP and Embedded server RF tcvr X Tiny Kernel Tiny flow drivers Application host s a s a svr sasa