1. Internet in a Box Building Distributed System Testbed in RAMP Zhangxi TanXiaofan Jiang Computer Science Department University of California, Berkeley Electrical Engineering Department University of California, Berkeley

2. 2 CS268 Spring 2006 Introduction Testing and debugging large scale distributed systems is difficult Problem with existing approaches  Scalability – O(100) nodes  Reproducibility - PlanetLab  Observability - Don’t know what’s going on inside  Cost, Space and Power Build distributed system testbed with over 1,000 nodes using multi modular FPGA system  Building block, HW/SW and networking infrastructure this semester

3. 3 CS268 Spring 2006 Methodology Intermediate platform – XUP  Fit 6/8 RISC processors per chip  Running uClinux with TCP/IP network  Software routers on chip  Sample distributed applications (Chord) Final platform - Research Accelerator for MultiProcessing (RAMP)  Scale up to 16 RISC (?) processors per chip x 4 = 64 cores

4. 4 CS268 Spring 2006 Previous Status Single processor running uClinux 2.4 on XUP board  Running @ 100 Mhz, 256M DDR  Zebra open source router (not fully tested)  Virtual NIC and point-to- point communication between processors Support polling and interrupt Firmware has been developed

5. 5 CS268 Spring 2006 Current Status Overview Multiple processors on one chip  6 cores on XUP running uClinux  Multiprocessor debugging and bootstrapping  XUP Clusters! Network Support  Ethernet Linux driver and TCP/IP network (Socket interface)  Network Interface Card (NIC) TCP/IP interconnect network  Transparent internal inter-core network and external inter-XUP network Demo application porting and performance evaluation  TTCP  Chord / i3 (after porting)  Web server w/CGI support

6. 6 CS268 Spring 2006 System Architecture

7. 7 CS268 Spring 2006 A XUP cluster

8. 8 CS268 Spring 2006 i3 / Chord / DHT

9. 9 CS268 Spring 2006 Available Applications / Utilities chmod df du find grep ifconfig iptables-restore lsmod netstat pwd route telnetd traceroute vi zcat agetty cmp dhclient echo flatfsd gunzip ifdown iptables- save mkdir nslookup rdate rsyslogd test true vmstat arp cp dhcpd egrep free gzip ifup kill modprobe null reboot sed tftp ttcp wc arping crond dhcrelay env ftp hd inetd killall more passwd reset sh thttpd umount wget basename crontab dmesg expand ftpd head init ln mount ping rm tail time uname which busybox date dpkg false ftpget hostname insmod login msh portmap rmdir tcpdump top uptime whoami cat dd dpkg-deb fgrep ftpput i3serverd iptables ls mv ps rmmod telnet touch version xargs Python

10. 10 CS268 Spring 2006 Challenges Porting apps is a painstaking job Configuration/Makefile problem Cross-compiling (Dynamic linking library, CPU type and etc.) Fork (using pthread library might be an alternative) More cores?  4-core version is quite stable now  6-core version (near 100% BRAM and Slices utilization) Hitting the power-wall without heat sink 100 Mhz is difficult Building and debugging is much more difficult Place and route time is unbearable! (floor plan tools required)

11. 11 CS268 Spring 2006 Future Work  8 cores?  Migrating design to RAMP (some dependencies)  And more…