1. Progress on System Architecture for Extreme Devices
Clusters
Massive Cluster
Gigabit Ethernet
Progress on System Architecture for Extreme Devices
David Culler
U.C. Berkeley
Endeavour Retreat
1/20/200

2. Outline The Very Large The Middle The Small
Millenium Cluster-of-cluster resources available to you
System Architecture
The Middle
Kiosks, laptops, and PDAs
The Small
Embedded Servers
Low power wireless
Architecture for Zillions of devices
1/20/2000
Endeavour Sys. Arch

3. Large Resource Deployment
In place:
Full 64-PIII Linux Cluster w/ Myrinet & ethernet (Gb rdy)
demonstrated on Ninja DDS
several remote 16-PIII clusters
DLIB 4-P + 1/4 TB cluster
1/2 TB storage server
Gb connection to dept, OC48 to NTON
Rootstock cluster dissemination facility
REXEC econ-based remote execution facility
100 KVA PDU
Demonstrated
intercampus network, routing, config (as per CNS)
full Gb/s with 2 source, 2 sinks
Deploying
45 x 4 of main cluster (2/4 GB mem, 18 GB disk, Gb ether)
1/20/2000
Endeavour Sys. Arch

4. Vineyard Cluster Architecture
dissemination, resource control, remote execution, and communication
Applications / Services (ISPACE/Kiosks)
Mgmt / Monitoring
PBS
I/O
MPI
VEXEC
TOOLS
REXEC
- VIA / GM, GbE
- Multicast
- NT / Linux (2.2.x)
- Stride Scheduler
Rootstock Distribution
1/20/2000
Endeavour Sys. Arch

5. Dissemination: Rootstock
2. Make the CS “graft”
- specify IP address
- pckg removes
- dchp, dns, nis,...
sanity check and build
- resolv.conf, /etc/hosts, ...
constructs cluster build (lease)
download CS build floppy
Cluster
3. CS power-on build
- xfer and localize DT
- add local admin scripts
- node build floppy
Cluster System
Distribution Center
leased
builds K
cluster
stock
- build
- os
- drvrs
- mill SW
- os mods
1. Cluster Stock
- Rootstock build pages
- Full Current Linux
- all fixes and pckgs
- SSL, SSH
- Cluster Drivers
- Cluster System Layers
- rexec, mpe, pbs
- Optional SW ($)
- Cluster Kernal Mods cs
4. Node power-on build
- local stock from CS IP
network
CAN
...
5. Cluster Update button (future)
- 2nd dialtone, CF engine, rolling update
1/20/2000
Endeavour Sys. Arch

6. REXEC / VEXEC
Resource Management, Autoconfig, Mechanism/Policy, Enforcement
Node A
Node B
Node C
Node D
rexecd
rexecd
rexecd
rexecd
Cluster IP Multicast Channel
vexecd (Policy A)
vexecd (Policy B)
“Nodes AB”
vexecd is a broker.
run indexer on Nodes AB at 3 credits/min
minimum $
rexec
%rexec –n 2 –r 3 indexer
1/20/2000
Endeavour Sys. Arch

7. “Intelligent” Middle
Deployed many laptops with , 3 base stations, many PDAs with IR
Solved the PDA to IR-dongle and PDA to Annex serial port (J. Hill)
Deployed two kiosks: touch-LCD, IR-ppp
act as server for managment
Demonstrated key aspects of the service infrastructure
eg: get device applet from service point
eSticky notes appln
motivated xcoding-security infrastucture
=> Need to harvest and extend
1/20/2000
Endeavour Sys. Arch

8. Small: Embedded Servers
Tested commercial products - promising
axis camera server
SOHO NAT, DHCP, Firewall server
plumbing = ethernet
Identified platform for building embedded servers
DIMM PC + Linux + …
MB RAM + 16 MB Flash Disk
1/20/2000
Endeavour Sys. Arch

9. Low-power Wireless Tested available options Selected RF monolithics
RF monolithics (used in Smart Rocks)
“virtual wire” is brain-dead, but good tranceiver
RadioMetrics (used in ISI RF Tags)
simple, primitive packet controller, no pwr down
World Wireless
nice MAC, but only infrastructure mode
Selected RF monolithics
working with BSAC and ISI on building-block
new packet controller + MAC
1/20/2000
Endeavour Sys. Arch

10. Zillions of Little Devices
Connected device as client well-established
distiller in the infrastructure spoonfeeds client
powerful services in power-limited devices!
How to get the illusion of continuous connectivity?
What about sensors-based devices?
they should behave as servers
eg: camera server
How to scale tiny server to need?
How to get illusion of continuous connectivity?
use the infrastructure
First a demonstration: note server in a PDA
1/20/2000
Endeavour Sys. Arch

11. Assumptions Computation and storage in the infrastructure is plentiful
Wired bandwidth is pervasive and essentially free
[ Multicast is widely accessible]
=> every device has a representative proxy in the infrastructure
1/20/2000
Endeavour Sys. Arch

12. Cautionary enabling thought
app
app
app
app
TCP IP
TCP protocol (acks, fsm, etc.) terminates in the kernel.
System specific protocol exists between kernel TCP agent and applications.
SSI clusters have another layer of network!
Think of device as an application (not big laptop)
Where is its kernel agent?
1/20/2000
Endeavour Sys. Arch

13. Device Access Architecture
low power
local device
link
Scalable, Available
Ninja Base AP
The Internet
Clients
persistent
named
representative
Physical
Device
Dev MC
Services
infra proxy provides name, state, queuing, etc.
extend toward AP as optimization
1/20/2000
Endeavour Sys. Arch

14. Demo Mapping The Internet Laptop w/ Ninja iSpace AP Laptop Browser
serial
persistent
named
representative
PDA
laptop
w/ pppd
TCP
BayStacker
Wireless
1/20/2000
Endeavour Sys. Arch

15. Key piece to build Low power controller with 2 stream devices
X = sensor + actuator for devices
X = host interface for AP and Embedded server
Application
Tiny Kernel
Tiny flow drivers X
RF tcvr
host a s s a
svr
s a
1/20/2000
Endeavour Sys. Arch