Mass RHIC Computing Facility Razvan Popescu - Brookhaven National Laboratory.

Slides:

Advertisements

Similar presentations

The RHIC-ATLAS Computing Facility at BNL HEPIX – Edinburgh May 24-28, 2004 Tony Chan RHIC Computing Facility Brookhaven National Laboratory.

Advertisements

Site Report: The Linux Farm at the RCF HEPIX-HEPNT October 22-25, 2002 Ofer Rind RHIC Computing Facility Brookhaven National Laboratory.

Computing Infrastructure

CASTOR Project Status CASTOR Project Status CERNIT-PDP/DM February 2000.

Vorlesung Speichernetzwerke Teil 2 Dipl. – Ing. (BA) Ingo Fuchs 2003.

IBM RS6000/SP Overview Advanced IBM Unix computers series Multiple different configurations Available from entry level to high-end machines. POWER (1,2,3,4)

What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s.

1 Andrew Hanushevsky - HEPiX, October 6-8, 1999 Mass Storage For BaBar at SLAC Andrew Hanushevsky Stanford.

PetaByte Storage Facility at RHIC Razvan Popescu - Brookhaven National Laboratory.

Backup Rationalisation Reorganisation of the CERN Computer Centre Backups David Asbury IT/DS Friday 6 December 2002.

Cluster computing facility for CMS simulation work at NPD-BARC Raman Sehgal.

The Mass Storage System at JLAB - Today and Tomorrow Andy Kowalski.

CERN - European Laboratory for Particle Physics HEP Computer Farms Frédéric Hemmer CERN Information Technology Division Physics Data processing Group.

CASPUR Site Report Andrei Maslennikov Sector Leader - Systems Catania, April 2001.

9/16/2000Ian Bird/JLAB1 Planning for JLAB Computational Resources Ian Bird.

The SLAC Cluster Chuck Boeheim Assistant Director, SLAC Computing Services.

CDF data production models 1 Data production models for the CDF experiment S. Hou for the CDF data production team.

Farm Management D. Andreotti 1), A. Crescente 2), A. Dorigo 2), F. Galeazzi 2), M. Marzolla 3), M. Morandin 2), F.

An Overview of PHENIX Computing Ju Hwan Kang (Yonsei Univ.) and Jysoo Lee (KISTI) International HEP DataGrid Workshop November 8 ~ 9, 2002 Kyungpook National.

Shigeki Misawa RHIC Computing Facility Brookhaven National Laboratory Facility Evolution.

Introduction to U.S. ATLAS Facilities Rich Baker Brookhaven National Lab.

November 2, 2000HEPiX/HEPNT FERMI SAN Effort Lisa Giacchetti Ray Pasetes GFS information contributed by Jim Annis.

POW : System optimisations for data management 11 November 2004.

GStore: GSI Mass Storage ITEE-Palaver GSI Horst Göringer, Matthias Feyerabend, Sergei Sedykh

1 U.S. Department of the Interior U.S. Geological Survey Contractor for the USGS at the EROS Data Center EDC CR1 Storage Architecture August 2003 Ken Gacke.

20-22 September 1999 HPSS User Forum, Santa Fe CERN IT/PDP 1 History  Test system HPSS 3.2 installation in Oct 1997 IBM AIX machines with IBM 3590 drives.

6/26/01High Throughput Linux Clustering at Fermilab--S. Timm 1 High Throughput Linux Clustering at Fermilab Steven C. Timm--Fermilab.

Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.

Developing & Managing A Large Linux Farm – The Brookhaven Experience CHEP2004 – Interlaken September 27, 2004 Tomasz Wlodek - BNL.

SLAC Site Report Chuck Boeheim Assistant Director, SLAC Computing Services.

KLOE Computing Update Paolo Santangelo INFN LNF KLOE General Meeting University of Rome 2, Tor Vergata 2002, December

Large Scale Parallel File System and Cluster Management ICT, CAS.

HPSS for Archival Storage Tom Sherwin Storage Group Leader, SDSC

JLAB Computing Facilities Development Ian Bird Jefferson Lab 2 November 2001.

CASTOR: CERN’s data management system CHEP03 25/3/2003 Ben Couturier, Jean-Damien Durand, Olof Bärring CERN.

O AK R IDGE N ATIONAL L ABORATORY U.S. D EPARTMENT OF E NERGY Facilities and How They Are Used ORNL/Probe Randy Burris Dan Million – facility administrator.

O AK R IDGE N ATIONAL L ABORATORY U.S. D EPARTMENT OF E NERGY Probe Plans and Status SciDAC Kickoff July, 2001 Dan Million Randy Burris ORNL, Center for.

Storage and Storage Access 1 Rainer Többicke CERN/IT.

8 October 1999 BaBar Storage at CCIN2P3 p. 1 Rolf Rumler BaBar Storage at Lyon HEPIX and Mass Storage SLAC, California, U.S.A. 8 October 1999 Rolf Rumler,

Test Results of the EuroStore Mass Storage System Ingo Augustin CERNIT-PDP/DM Padova.

CERN IT Department CH-1211 Genève 23 Switzerland t Frédéric Hemmer IT Department Head - CERN 23 rd August 2010 Status of LHC Computing from.

ATLAS Tier 1 at BNL Overview Bruce G. Gibbard Grid Deployment Board BNL 5-6 September 2006.

PHENIX Computing Center in Japan (CC-J) Takashi Ichihara (RIKEN and RIKEN BNL Research Center ) Presented on 08/02/2000 at CHEP2000 conference, Padova,

The KLOE computing environment Nuclear Science Symposium Portland, Oregon, USA 20 October 2003 M. Moulson – INFN/Frascati for the KLOE Collaboration.

PC clusters in KEK A.Manabe KEK(Japan). 22 May '01LSCC WS '012 PC clusters in KEK s Belle (in KEKB) PC clusters s Neutron Shielding Simulation cluster.

Status SC3 SARA/Nikhef 20 juli Status & results SC3 throughput phase SARA/Nikhef Mark van de Sanden.

HIGUCHI Takeo Department of Physics, Faulty of Science, University of Tokyo Representing dBASF Development Team BELLE/CHEP20001 Distributed BELLE Analysis.

CERN Computer Centre Tier SC4 Planning FZK October 20 th 2005 CERN.ch.

Randy MelenApril 14, Stanford Linear Accelerator Center Site Report April 1999 Randy Melen SLAC Computing Services/Systems HPC Team Leader.

RHIC/US ATLAS Tier 1 Computing Facility Site Report Christopher Hollowell Physics Department Brookhaven National Laboratory HEPiX Upton,

January 30, 2016 RHIC/USATLAS Computing Facility Overview Dantong Yu Brookhaven National Lab.

May 10, 2000PHENIX CC-J Updates1 PHENIX CC-J Updates - Preparation For Opening - N.Hayashi / RIKEN May 10, 2000 PHENIX Computing

W.A.Wojcik/CCIN2P3, Nov 1, CCIN2P3 Site report Wojciech A. Wojcik IN2P3 Computing Center URL:

Batch Software at JLAB Ian Bird Jefferson Lab CHEP February, 2000.

Mass Storage at SARA Peter Michielse (NCF) Mark van de Sanden, Ron Trompert (SARA) GDB – CERN – January 12, 2005.

Tackling I/O Issues 1 David Race 16 March 2010.

1 5/4/05 Fermilab Mass Storage Enstore, dCache and SRM Michael Zalokar Fermilab.

Oct. 6, 1999PHENIX Comp. Mtg.1 CC-J: Progress, Prospects and PBS Shin’ya Sawada (KEK) For CCJ-WG.

W.A.Wojcik/CCIN2P3, HEPiX at SLAC, Oct CCIN2P3 Site report Wojciech A. Wojcik IN2P3 Computing Center URL:

NASA Langley Research Center’s Distributed Mass Storage System (DMSS) Juliet Z. Pao Guest Lecturing at ODU April 8, 1999.

Bernd Panzer-Steindel CERN/IT/ADC1 Medium Term Issues for the Data Challenges.

12/19/01MODIS Science Team Meeting1 MODAPS Status and Plans Edward Masuoka, Code 922 MODIS Science Data Support Team NASA’s Goddard Space Flight Center.

High Performance Storage System (HPSS) Jason Hick Mass Storage Group HEPiX October 26-30, 2009.

Video Security Design Workshop:

PC Farms & Central Data Recording

Direct Attached Storage Overview

Research Data Archive - technology

San Diego Supercomputer Center

CASTOR: CERN’s data management system

Lee Lueking D0RACE January 17, 2002

Presentation transcript:

Mass RHIC Computing Facility Razvan Popescu - Brookhaven National Laboratory

SLAC -- October 1999Mass RCF2 Overview t Data Types: –Raw: very large volume (xPB), average bandwidth (50MB/s). –DST: average volume (x00TB), large bandwidth (x00MB/s). –mDST: low volume (x0TB), large bandwidth (x00MB/s).

SLAC -- October 1999Mass RCF3 Data Flow (generic) RHIC File Servers (DST/mDST) Reconstruction Farm (Linux) Analysis Farm (Linux) Archive (HPSS) raw DST mDST DST 35MB/s 50MB/s 10MB/s 200MB/s 400MB/s

SLAC -- October 1999Mass RCF4

SLAC -- October 1999Mass RCF5 Present resources t Tape Storage: –(1) STK Powderhorn silo (6000 cart.) –(11) SD-3 (Redwood) drives. –(10) 9840 (Eagle) drives. t Disk Storage: –~8TB of RAID disk. 1TB for HPSS cache. 7TB Unix workspace. t Servers: –(5) RS/6000 H50/70 for HPSS. –(6) E450&E4000 for file serving and data mining.

SLAC -- October 1999Mass RCF6 The HPSS Archive t Constraints - large capacity & high bandwidth: –Two types of tape technology: SD-3 (best $/GB) & 9840 (best $/MB/s). –Two tape layers hierarchies. Easy management of the migration. t Reliable and fast disk storage: –FC attached RAID disk. t Platform compatible with HPSS: –IBM, SUN, SGI.

SLAC -- October 1999Mass RCF7 HPSS Structure t (1) Core Server: –RS/6000 Model H50 –4x CPU –2GB RAM –Fast Ethernet (control) –Hardware RAID (metadata storage)

SLAC -- October 1999Mass RCF8 HPSS Structure t (3) Movers: –RS/6000 Model H70 –4x CPU –1GB RAM –Fast Ethernet (control) –Gigabit Ethernet (data) (1500&9000MTU) –2x FC attached RAID - 300GB - disk cache –(3-4) SD-3 “Redwood” tape transports –(3-4) 9840 “Eagle” tape transports

SLAC -- October 1999Mass RCF9 HPSS Structure t Guarantee availability of resources for a specific user group separate resources separate PVRs & movers. t One mover per user group total exposure to single-machine failure. t Guarantee availability of resources for Data Acquisition stream separate hierarchies. t Result: 2PVR&2COS&1Mvr per group.

SLAC -- October 1999Mass RCF10 HPSS topology M3M2M1Core Net 2 - Control (100baseT) Net 1 - Data (1000baseSX) STK 10baseT N x PVR pftpd Client (Routing)

SLAC -- October 1999Mass RCF11 HPSS Performance t 80 MB/sec for the disk subsystem. t 1 CPU per 40MB/sec for TCPIP (Gbit) traffic (1500MTU). t ~8MB/sec per SD-3 transport. t ? per 9840 transport.

SLAC -- October 1999Mass RCF12 I/O intensive systems t Mining and Analysis systems. t High I/O & moderate CPU usage. t To avoid large network traffic merge file servers with HPSS movers: –Major problem with HPSS support on non-AIX platforms. –Several (Sun) SMP machines or Large (SGI) Modular System.

SLAC -- October 1999Mass RCF13 I/O intensive systems t (6) NFS file servers for workareas –(5) x E450 + (1) x E4000 –4(6) x CPU; 2GB RAM; Fast/Gbit Ethernet. –2 x FC attached hardware RAID - 1.5TB t (1) NFS Home directory server (E450). t (3+3) AFS Servers (code dev. & home dirs.) –RS/6000 model E30 and 43P t (NFS to AFS migration)

SLAC -- October 1999Mass RCF14 Problems t Short lifecycle of the SD-3 heads. –~ 500 hours < 2 average usage. (6 of 10 drives in 10 months) t Low throughput interface (F/W) for SD-3 -> high slot consumption. t 9840 ??? t HPSS: tape cartridge transport error. –Built a monitoring tool to try to predict transport failure (based of soft error frequency). t SFS response when heavy loaded - no graceful failure (timeouts & lost connections).

SLAC -- October 1999Mass RCF15 Issues t Partially tested two tape layer hierarchies: –Cartridge based migration. –Manually scheduled reclaim. t Integration of file server and mover functions on the same node: –Solaris mover port. –Not an objective anymore.

SLAC -- October 1999Mass RCF16 Issues t Guarantee avail. of resources for specific user groups: –Separate PVRs & movers. –Total exposure to single-mach. failure ! t Reliability: –Distribute resources across movers share movers (acceptable?). –Inter-mover traffic: 1 CPU per 40MB/sec TCPIP per adapter: Expensive!!!

SLAC -- October 1999Mass RCF17 Inter-mover traffic - solutions t Affinity. –Limited applicability. t Diskless hierarchies. –Not for SD-3. Not tested on t High performance networking: SP switch. –IBM only. t Lighter protocol: HIPPI. –Expensive hardware. t Multiply attached storage (SAN). –Requires HPSS modifications.

SLAC -- October 1999Mass RCF18 Multiply Attached Storage (SAN) Mover 1Mover 2 Client (!) 1 2

SLAC -- October 1999Mass RCF19 Summary t Problems with divergent requirements: –Cost effective archive capacity and bandwidth. Two tape hierarchies: SD-3 & Test the configuration. –Availability and reliability of HPSS resources. Separated COS and shared movers. Inter-mover traffic ?!? t Merger of file servers and HPSS movers?