1. O AK R IDGE N ATIONAL L ABORATORY U.S. D EPARTMENT OF E NERGY Probe Plans and Status SciDAC Kickoff July, 2001 Dan Million Randy Burris ORNL, Center for Computational Sciences

2. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Overview  Proposal’s plans for Probe involvement  HPSS overview  Probe as a “place to be”  SciDAC applications with Probe connections  Technology on hand

3. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Plans as stated in the proposal  Five research thrusts  High-bandwidth transfers of very large datasets Optimize blocksizes and configurations TCP-friendly UDP service Test AIX 5L for improvements in TCP/IP stack Evaluate ATM’s effect on TCP Performance Web100  Enhancements to HSI (Hierarchical Storage Interface)  Evaluate Gigabyte System Network and trunked Ethernet  Investigate new protocols (ST, Storage over IP)  Adapt and utilize simulation model of HPSS  Provide “place to be” – testbed for other SDM ETC activities

4. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY HPSS overview  High Performance Storage System  Targets petabyte capacities and gigabyte/second transfer rates  Core servers manage metadata and control movers  Movers communicate over data network and move data from network to storage and vice versa  Distributed architecture – core servers and movers can be replicated and run on hundreds of nodes

5. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY HPSS overview (continued)  ORNL is a development partner  Can establish HPSS systems without limit (4 now)  Production  HPSS/AIX, most current version  HPSS/Sun, next version  HPSS/AIX, next version  Can modify HPSS itself

6. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Probe – “Place to be” Overview of ORNL Probe Cell, June 2001 Stingray RS/6000 S80 Marlin RS/6000 H70 STK Silo 300 GB SCSI RAID Disks Sun E250 Compaq DS20 360 GB Sun FibreChannel Disks 360 GB STK FibreChannel Disks FibreChannel Switch GSN Switch Origin 2000 Reality Monster RS/6000 B80 External Esnet Router To NERSC Probe Sun Ultra 10 STK Silo IBM SP Compaq Sierra 3494 Library GSN Bridge RS/6000 44P-170 Probe Shared with Production Intel Dual Xeon Linux Sun E450 IBM F50 SGI Origin 200 SN6000 Gigabit Ethernet

7. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY ORNL Probe Cell, June 2001 – equipment for agent infrastructure Stingray RS/6000 S80 Marlin RS/6000 H70 Origin 2000 Reality Monster RS/6000 B80 External Esnet Router To NERSC Probe IBM SP Compaq Sierra Probe Shared with Production Intel Dual Xeon Linux Sun E450 Gigabit Ethernet Jumbo Agents might run on any machine; variety of platforms shown. CPU intensive agents might run on Stingray or some supercomputer. Sun E250 Compaq DS20 IBM F50 SGI Origin 200

8. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY ORNL Probe Cell, June 2001 – equipment for file-transfer work Stingray RS/6000 S80 Marlin RS/6000 H70 STK Silo 300 GB SCSI RAID Disks 360 GB Sun FibreChannel Disks GSN Switch Origin 2000 Reality Monster RS/6000 B80 External Esnet Router To NERSC Probe Gigabit Ethernet Jumbo STK Silo IBM SP Compaq Sierra 3494 Library GSN Bridge Probe Shared with Production Intel Dual Xeon Linux Sun E450 SN6000 Variety of hosts Variety of storage equipment Variety of links Sun E250 Compaq DS20 360 GB STK FibreChannel Disks FibreChannel Switch IBM F50 SGI Origin 200

9. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY ORNL Probe Cell, June 2001 – equipment for data research Stingray – S80 6-processor RS/6000 2GB Excellent CPU and I/O capability Marlin – H70 4-processor RS/6000 2 GB Oracle, DB2 STK Silo 300 GB SCSI RAID Disks 360 GB Sun FibreChannel Disks 360 GB STK FibreChannel Disks FibreChannel Switch External Esnet Router To NERSC Probe Gigabit Ethernet Jumbo STK Silo IBM SP Compaq Sierra 3494 Library Probe Shared with Production Intel Dual Xeon Linux Sun E450 Oracle, DB2 HPSS CPU intensive work might run on Stingray, Marlin or some supercomputer. Available: HPSS/IBM, HPSS/Solaris, Oracle 8i and DB2 HPSS

10. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Application involvements with Probe  Climate  Goal: production use of WAN bulk transfers  Probe provides: Some storage space (source of data) Access to new transfer mechanism and associated equipment Development and testing to expedite their work  Astrophysics  Goal: high-bandwidth transfers between heterogeneous platforms  Probe expects to provide: ~300 GB storage (disk and tape) Access to S80 or perhaps other machines Access to GSN equipment Assistance using the above

11. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Climate: Wide-Area bulk transfers  WAN bulk transfers are a vital element of SDM ETC  Wide-area bulk data transfer requires:  High sustained rate  Asynchronous transfer (so user isn’t tied up)  Suitability for many source/sink pairs  “Remote Mover” architecture works  Designed for Climate – one direction, one source/sink pair  Not suitable for SciDAC needs  Considering implementing a front-end for HPSS  Currently testing ESnet III  ORNL/NERSC performance poor (bandwidth quadrupled but latency doubled; other problems too)

12. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY ORNL GSN Configuration ODS/Essential GSN Switch 6 ports Genroco Bridge Stingray RS/6000 Model S80 AIX 4.3.3 Capsicum Origin 2000 Reality Monster IRIX 6.5.10 Compaq Alpha DS20 Tru64 4.0f 2 Gigabit Ethernet blades 2 FibreChannel blades Cave Visualization Equipment Sun, Clariion or Compaq disks IBM, Compaq use Genroco 6466 NICs (64 bit 66 MHz) Marlin RS/6000 Model H70 (50 MHz slots) Storage Equipment Earl RS/6000 Model B80 (50 MHz slots) Alternative nodes

13. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY ORNL Probe Cell, June 2001 – Possible equipment for supernova visualization Stingray RS/6000 S80 STK Silo 300 GB SCSI RAID Disks Compaq DS20 360 GB Sun FibreChannel Disks 360 GB STK FibreChannel Disks FibreChannel Switch GSN Switch Origin 2000 Reality Monster RS/6000 B80 External Esnet Router To NERSC Probe Gigabit Ethernet Jumbo STK Silo IBM SP Compaq Sierra 3494 Library GSN Bridge ORNL Backbone Probe Shared with Production Sun E450 AIX 5L, GSN SGI Origin 200 GSN RAIT HPSS SN6000 Marlin RS/6000 H70 HPSS, GSN CPU intensive work might run on Stingray, Marlin or some supercomputer.

14. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Technology on hand and available  On hand  HPSS (unlimited instantiations) and HPSS development license  ESnet III OC12  Oracle 8i and DB2 (current developer’s editions)  Inter-HPSS hsi application  TCP-friendly UDP service  OPNET modeling product  Variety of current storage and networking hardware and software  RAIT (scheduled for trial in August)  In plan:  ORNL GRID node (MICS funded)  Web100 participation (MICS funded)  MPI/IO software

15. OAK RIDGE NATIONAL LABORATORY U.S. DEPARTMENT OF ENERGY Questions?