SJ – June 2003 1 CERN openlab for DataGrid applications Sverre Jarp CERN openlab CTO IT Department, CERN.

Slides:



Advertisements
Similar presentations
Bernd Panzer-Steindel, CERN/IT WAN RAW/ESD Data Distribution for LHC.
Advertisements

Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up.
Status GridKa & ALICE T2 in Germany Kilian Schwarz GSI Darmstadt.
GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.
Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.
Bernd Panzer-Steindel, CERN/IT 2 * 50 Itanium Server (dual 1.3/1.5 GHz Itanium2, 2 GB mem) High Througput Prototype (openlab + LCG prototype) (specific.
1 A Basic R&D for an Analysis Framework Distributed on Wide Area Network Hiroshi Sakamoto International Center for Elementary Particle Physics (ICEPP),
Status Report on Tier-1 in Korea Gungwon Kang, Sang-Un Ahn and Hangjin Jang (KISTI GSDC) April 28, 2014 at 15th CERN-Korea Committee, Geneva Korea Institute.
ALICE Data Challenge V P. VANDE VYVRE – CERN/PH LCG PEB - CERN March 2004.
CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University30/1/2003 CMS Status and Plans Progress towards GridPP milestones Workload.
23 Oct 2002HEPiX FNALJohn Gordon CLRC-RAL Site Report John Gordon CLRC eScience Centre.
May 2004Sverre Jarp1 Preparing the computing solutions for the Large Hadron Collider (LHC) at CERN Sverre Jarp, openlab CTO IT Department, CERN.
March 2003 CERN 1 EDG and AliEn in Prague Dagmar Adamova INP Rez near Prague.
ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID Project Phase2 of the ScotGRID Project Phase2 of the.
The ALICE DAQ: Current Status and Future Challenges P. VANDE VYVRE CERN-EP/AID.
PDSF at NERSC Site Report HEPiX April 2010 Jay Srinivasan (w/contributions from I. Sakrejda, C. Whitney, and B. Draney) (Presented by Sandy.
RAL Site Report Andrew Sansum e-Science Centre, CCLRC-RAL HEPiX May 2004.
10/22/2002Bernd Panzer-Steindel, CERN/IT1 Data Challenges and Fabric Architecture.
NL Service Challenge Plans Kors Bos, Sander Klous, Davide Salomoni (NIKHEF) Pieter de Boer, Mark van de Sanden, Huub Stoffers, Ron Trompert, Jules Wolfrat.
Grid User Interface for ATLAS & LHCb A more recent UK mini production used input data stored on RAL’s tape server, the requirements in JDL and the IC Resource.
CERN IT Department CH-1211 Genève 23 Switzerland t Frédéric Hemmer IT Department Head - CERN 23 rd August 2010 Status of LHC Computing from.
The CERN openlab for DataGrid Applications A Practical Channel for Collaboration François Fluckiger.
SJ – Mar The “opencluster” in “openlab” A technical overview Sverre Jarp IT Division CERN.
CERN openlab II (2006 – 2008) Grid-related activities Sverre Jarp CERN openlab CTO sverre.jarp at cern.ch.
BNL Service Challenge 3 Status Report Xin Zhao, Zhenping Liu, Wensheng Deng, Razvan Popescu, Dantong Yu and Bruce Gibbard USATLAS Computing Facility Brookhaven.
CERN Computer Centre Tier SC4 Planning FZK October 20 th 2005 CERN.ch.
CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,
Status of India CMS Grid Computing Facility (T2-IN-TIFR) Rajesh Babu Muda TIFR, Mumbai On behalf of IndiaCMS T2 Team July 28, 20111Status of India CMS.
SJ – Nov CERN’s openlab Project Sverre Jarp, Wolfgang von Rüden IT Division CERN 29 November 2002.
Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.
EGEE is a project funded by the European Commission under contract IST NA4/HEP work F Harris (Oxford/CERN) M.Lamanna(CERN) NA4 Open meeting.
LCG Service Challenges SC2 Goals Jamie Shiers, CERN-IT-GD 24 February 2005.
Mass Storage at SARA Peter Michielse (NCF) Mark van de Sanden, Ron Trompert (SARA) GDB – CERN – January 12, 2005.
ASCC Site Report Eric Yen & Simon C. Lin Academia Sinica 20 July 2005.
BNL dCache Status and Plan CHEP07: September 2-7, 2007 Zhenping (Jane) Liu for the BNL RACF Storage Group.
CERN News on Grid and openlab François Fluckiger, Manager, CERN openlab for DataGrid Applications.
GDB Meeting 12. January Bernd Panzer-Steindel, CERN/IT 1 Mass Storage at CERN GDB meeting, 12. January 2005.
CNAF - 24 September 2004 EGEE SA-1 SPACI Activity Italo Epicoco.
Bernd Panzer-Steindel CERN/IT/ADC1 Medium Term Issues for the Data Challenges.
BeStMan/DFS support in VDT OSG Site Administrators workshop Indianapolis August Tanya Levshina Fermilab.
J Jensen / WP5 /RAL UCL 4/5 March 2004 GridPP / DataGrid wrap-up Mass Storage Management J Jensen
18/12/03PPD Christmas Lectures 2003 Grid in the Department A Guide for the Uninvolved PPD Computing Group Christmas Lecture 2003 Chris Brew.
ALICE Computing Data Challenge VI
OpenCluster Planning Sverre Jarp IT Division CERN October 2002.
Real Time Fake Analysis at PIC
The Beijing Tier 2: status and plans
“A Data Movement Service for the LHC”
Status Report on LHC_2 : ATLAS computing
OpenLab Enterasys Meeting
NL Service Challenge Plans
LCG 3D Distributed Deployment of Databases
LCG Deployment in Japan
CERN openlab for DataGrid applications Programme of Work Overview F
LHC experiments Requirements and Concepts ALICE
Database Readiness Workshop Intro & Goals
Grid related projects CERN openlab LCG EDG F.Fluckiger
ALICE Physics Data Challenge 3
Bernd Panzer-Steindel, CERN/IT
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
The INFN TIER1 Regional Centre
openLab Technical Manager
Conditions Data access using FroNTier Squid cache Server
The INFN Tier-1 Storage Implementation
The CERN openlab for DataGrid Applications François Fluckiger
Grid Canada Testbed using HEP applications
LHC Data Analysis using a worldwide computing grid
Porting LCG to IA64 Andreas Unterkircher CERN openlab May 2004
CERN openlab for DataGrid applications Setting the Scene F
The LHCb Computing Data Challenge DC06
Presentation transcript:

SJ – June CERN openlab for DataGrid applications Sverre Jarp CERN openlab CTO IT Department, CERN

SJ – June CERN openlab LCG Department’s main R&D focus Framework for collaboration with industry Evaluation, integration, validation of cutting-edge technologies Initially a 3-year lifetime Later: Annual renewals CERN openlab

SJ – June Openlab sponsors 5 current partners Enterasys: 10 GbE core routers HP: Integrity servers (103 * 2-ways, 2 * 4-ways) Two fellows (co-sponsored with CERN) IBM: Storage Tank file system (SAN FS) w/metadata servers and data servers (currently with 28 TB) Intel: 64-bit Itanium processors & 10 Gbps NICs Oracle: 10g Database software w/add-on’s Two fellows One contributor Voltaire 96-way Infiniband switch

SJ – June The opencluster in its new position in the Computer Centre

SJ – June GE 20 Tape Server 56 IA64 Server (1.3/1.5 GHz Itanium2, 2 GB mem) 180 IA32 CPU Server (2.4 GHz P4, 1 Gb mem) 28 IA32 Disk Server (~ 1TB disk space each) multi GE connections to the backbone 10GE WAN connection GE per node 10GE per node GE per node New High Througput Prototype (  Feb. 2004) ENTERASYS N7 Integration with the LCG testbed

SJ – June Recent achievements (selected amongst many others) Hardware and software Key ingredients deployed in Alice Data Challenge V Internet2 land speed record between CERN and CalTech Porting and verification of CERN/HEP software on 64-bit architecture CASTOR, ROOT, CLHEP, GEANT4, ALIROOT, etc. Parallel ROOT data analysis Port of LCG software to Itanium

SJ – June Trigger Level 0,1 Trigger Level 2 High-Level Trigger Transient Data Storage (TDS) Event-Building Network Storage network Detector Digitizers Front-end Pipeline/Buffer Decision Readout Buffer Decision Sub-event Buffer Local Data Concentrators (LDC) Event Buffer Global Data Collectors (GDC) Permanent Data Storage (PDS) Decision Detector Data Link (DDL) Data ADC V - Logical Model and Requirements Tested during ADC 25 GB/s 2.50 GB/s 1.25 GB/s

SJ – June Achievements (as seen by Alice) Sustained bandwidth to tape: Peak 350 MB/s Reached production-quality level only last week of testing Sustained 280 MB/s over 1 day but with interventions [goal was 300] IA-64 from openlab successfully integrated in the ADC V Goal for ADC VI: 450 MB/s

SJ – June Gbps WAN tests Initial breakthrough during Telecom-2003 with IPv4 (single/multiple) streams: 5.44 Gbps Linux, Itanium-2 (RX 2600), Intel 10Gbps NIC Also IPv6 (single/multiple) streams In February Again IPv4, but multiple streams (DataTag + Microsoft): 6.25 Gbps Windows/XP, Itanium-2 (Tiger-4), S2IO 10 Gbps NIC In June (not yet submitted) Again IPv4, and single stream (Datatag + Openlab): 6.55 Gbps Linux, Itanium-2 (RX2600), S2IO NIC openlab still has a slightly better result than NewiSys Opteron 4-way box and a heavily tuned Windows/XP

SJ – June Cluster parallelization Parallel ROOT Facility (PROOF): Excellent scalability with 64 processors last year Tests in progress for 128 (or more) CPUs MPI software installed Ready for tests with BEAMX (similar program to Sixtrack) Alinghi software also working Collaboration with team at EPFL Uses Ansys CFX distcc installed and tested Compilation time reduced both for GNU and Intel compiler

SJ – June Gridification A good success story: Starting point: The software chosen for LCG (VDT + EDG) had been developed only with IA32 (and specific Red Hat versions) in mind Consequence: Configure-files and make-files not prepared for multiple architectures. Source files not available in distributions (often not even locatable) Stephen Eccles, Andreas Unterkircher worked for many months to complete the porting of LCG-2 Result: All major components now work on Itanium/Linux: Worker Nodes, Compute Elements, Storage Elements, User Interface, etc. Tested inside EIS Test Grid Code, available via Web-site, transferred to HP sites (Initially Puerto Rico and Bristol) Changes given back to developers VDT now built also for Itanium systems Porting experience summarized in white paper (on the Web) From now on the Grid is heterogeneous!

SJ – June Storage Tank Random Access test (mid-March) Scenario: 100 GB dataset, randomly accessed in ~50kB blocks 1 – GHz P4-class clients, running 3 – “jobs” Hardware 4 IBM x335 metadata servers 8 IBM 200i controllers, 336 SCSI disks Added 2 IBM x345 servers as disk controllers after the test Results Peak data rate: 484 MB/s (with 9855 simultaneous “jobs”) After the test, special tuning, 10 servers, smaller number of clients: 705 MB/s Ready to be used in Alice DC VI

SJ – June Next generation disk servers Based on state-of-the-art equipment: 4-way Itanium server (RX4640) Two full-speed PCI-X slots 10 GbE and/or Infiniband Two 3ware 9500 RAID controllers In excess of 400 MB/s RAID-5 read speed Only 100 MB/s for write w/RAID MB/s RAID 0 24 * S-ATA disks with 74 GB WD740 10k rpm Burst speed of 100 MB/s Goal: Saturate 10GbE card for reading (at least 500 MB/s with standard MTU and 20 streams). Writing as fast as possible.

SJ – June Tests (initially) between CERN and Fermilab + NIKHEF Multiple HP Itanium servers with dual NICs Disk to disk transfers via GridFTP Each server: 100 MB/s IN MB/s OUT Aggregation of multiple streams across 10 GbE link Similar tuning as Internet2 tests Possibly try the 4-way 10GbE server and Enterasys X-series router RAL IN2P3 BNL FZK CNAF USC PIC ICEPP FNAL NIKHEF Krakow Taipei CIEMAT TRIUMF Rome CSCS Legnaro UB IFCA IC MSU Prague Budapest Cambridge Data distribution ~70 Gbits/sec Data export to LCG Tier-1/-2 “Service Data Challenge” Stability is paramount – no longer just “raw” speed

SJ – June Conclusions CERN openlab: Solid collaboration with our industrial partners Encouraging results in multiple domains We believe sponsors are getting good “ROI” But only they can really confirm it No risk of running short of R&D IT Technology is still moving at an incredible pace Vital for LCG that the “right” pieces of technology are available for deployment Performance, cost, resilience, etc. 6 students, 4 fellows