Southern California Infrastructure Philip Papadopoulos Greg Hidley

UCSD Packet Test Bed OptIPuter Year 2

Goals Expand our Campus-wide Research Instrument –Support of Researcher Needs – Focus on Application Needs –Deployment of Scalable Endpoints –Continued Evolution of the Packet-Based OptIPuter –Begin Deployment of 10 GigE Technologies –Drive Towards Goal of 1:1 Bisection Bandwidth –5:1 For This Year Evaluation of Parallel Storage Systems with Remote Access Expand the UCSD-based OptIPuter from Campus-only to Southern California and Eventually to Chicago

Year 2 Accomplishments Staffing –Hired UCSD Optiputer Project Manager Aaron Chin –Additional 1.5 FTE Effort Towards Infrastructure Deployment: –Max Okumoto, Sean O’Connell, David Hutches, Mason Katz Node Buildout –48 Node (IA-32), 21 TB (300 Spindles) IBM Storage Cluster (JSOE) –128 Node (IA-32) Sun Compute Cluster (SDSC) –22 Node (Opteron) Sun Viz Cluster (SOM) Geowall2 –3 Node (IA-32) Shuttle Viz Cluster (CRCA) Display Wall –Expected By 10/1/04 –3 Sun (Opteron) Clusters, Size to be Determined Network Buildout –1 Gigabit Transport –Connectivity To Campus VLAN Infrastructure And Border Router (T320) –10 Gigabit Transport –Single Interface for Chiaro –Extreme 400 (48 Port Gige with two 10gige Uplinks) Layer 2 Fabric –1 Each at JSOE, CSE and SDSC –5:1 Bisection for the Storage Cluster

Making The Campus-Wide OptIPuter Usable Solving Network Issues –Converted to Public IP Address Space to Facilitate Off-campus Connectivity –4 GigE Channel Bonding – Worked With Chiaro to Improve Performance –Load-balancing Algorithm Among Links is Handled Differently at Layer 2 (Dell) and Layer 3 (Chiaro) –35% of Bonded Link Capacity Before “Fix”, 60% Afterwards. L2-L2 (Dell-to- Dell) is 95% For Comparison –PVFS –8 IBM Storage Nodes Running as PVFS Servers –Clients Running at SIO, JSOE and SOM –Performance Reasonable but not Significantly Larger than 1 Gigabit Developing Support for Users [ ] –Account Request Tools –Network Monitoring Facilities –Technical Network Configuration Information –Node Configuration Information –IP Allocation Map

SoCal CalREN-XD OptIPuter Build-Out Expected Completion July 2004

Expanding the OptIPuter LambdaGrid Allows us to build OptIPuters at larger diameters (latency metric) Significant enabler for investigating Hybrid Networks (packet + circuit)

Year 3 Plans: Enhance Campus OptIPuter A Substantial Portion of the Physical Build Completes in Year 2 –Endpoints, Cross-campus Fiber, Commodity Endpoints Increase Campus Bandwidth –Work Towards More Extensive 10GigE Integration –Optiputer HW Budget Limited In Year 3, Focus is on Network Extension –Connect Two Campus Sites with 32-node Clusters At 10GigE –3:1 Campus Bisection Ratio Add/Expand a Moderate Number of new Campus Endpoints –Add New Endpoints Into The Chiaro Network –UCSD Sixth College –JSOE (Engineering) Collaborative Visualization Center –New Calit2 Research Facility –Add 3 General-purpose Sun Opteron Clusters at Key Campus Sites (Compute and Storage); Clusters Will All Have PCI-X (100 Mhz, 1Gbps) –Deploy Infiniband on Our IBM Storage Cluster and on a Previously-Donated Sun 128-node Compute Cluster Complete Financial Acquisition of the Chiaro Router

Year Three Goals Integrate New NSF Quartzite MRI Goal -- integration of Packet-based (SoCal) and Circuit-based (Illinois) Approaches a Hybrid System –Add Additional O-O-O Switching Capabilities Through a Commercial (Calient Or Glimmerglass) All-optical Switch and the Lucent (Pre- commercial) Wavelength Selective Switch –Begin CWDM Deployment to Extend Optical Paths Around UCSD and Provide Additional Bandwidth –Add Additional 10GigE in Switches and Cluster Node NICs MRI Proposal (Quartzite, Recommended for Funding) Allows Us to Match the Network to the Number of Existing Endpoints This is a New Kind of Distributed Instrument –300+ Components Distributed Over the Campus –Simple and Centralized Control for Other Optiputer Users

UCSD Quartzite Core at Completion (Year 5 of OptIPuter) Recommended for Funding Physical HW to Enable Optiputer and Other Campus Networking Research Hybrid Network Instrument