ScotGRID Report: Prototype for Tier-2 Centre for LHC Akram Khan On Behalf of the ScotGRID Team ( Akram Khan On Behalf of the ScotGRID Team (

GridPP6 Collaboration MeetingScotGRID Report Overview of Talk Misc Bits Summary & Outlook Future Plans Hardware / Operation What are we hoping to do..?

GridPP6 Collaboration MeetingScotGRID Report Never Forget The Spirit of the Project The LHC Computing Challenge for Scotland 2000: JREI Bid The JREI funds will make possible to commission and fully exercise a prototype LHC computing centre in Scotland The Centre would provide: 1. Technical service based for the grid(GIIS, VO services…) 2. DataStore to handle samples of data towards part. Analysis 3. Significant simulation production capability 4. Excellent network connection RAL + regional sites 5. Support grid middle devel. with CERN and RAL 6. Support core software devel. within LHCb and ATLAS 7. Support user applications in other scientific areas This will enable us to answer:  Is the grid viable solution for LHC computing challenge  Can a two-site Tier-2 centre be setup and operate effectively  How can network topology between Ed,GL, RAL & CERN

GridPP6 Collaboration MeetingScotGRID Report ScotGRID: Glasgow / Edinburgh  59 x330 dual PIII 1GHz/2 Gbyte compute nodes  2 x340 dual PIII/1 GHz /2 Gbyte head nodes  3 x340 dual PIII/1 GHz/2 Gbyte storage nodes, each with 11 by 34 Gbytes in Raid 5  1 x340 dual PIII/1 GHz/0.5 Gbyte masternode  59 x330 dual PIII 1GHz/2 Gbyte compute nodes  2 x340 dual PIII/1 GHz /2 Gbyte head nodes  3 x340 dual PIII/1 GHz/2 Gbyte storage nodes, each with 11 by 34 Gbytes in Raid 5  1 x340 dual PIII/1 GHz/0.5 Gbyte masternode  xSeries quad Pentium Xeon 700 MHz/16 Gbytes, server  1 FAStT 500 controller  7 diskarrays of 10 x 73 Gb disk  xSeries quad Pentium Xeon 700 MHz/16 Gbytes, server  1 FAStT 500 controller  7 diskarrays of 10 x 73 Gb disk

GridPP6 Collaboration MeetingScotGRID Report ScotGRID - Glasgow

GridPP6 Collaboration MeetingScotGRID Report ScotGRID: Glasgow - Schematic Internet VLAN VLAN 100 Mbps 1000 Mbps MasternodeStorage NodesHead Nodes Compute Nodes Campus Backbone bottleneck

GridPP6 Collaboration MeetingScotGRID Report ScotGRID: Edinburgh - Schematic Disk Arrays(Total 4.6 Tb) FastT 500 Storage Controller Server (4*Pentium Xeon, 16Gb RAM) SRIF Network

GridPP6 Collaboration MeetingScotGRID Report Towards a Prototype Tier Q1 Q2 Q3 Q Q4 PrototypesPrototypes xCAT tutorial, attempt on masternode ScotGRID room handed over to builders Building work complete xCAT reinstall User registration, trail production Installation of Software Configuring disk array Reconfiguring kernel drivers for FAStT storage controller User registration, Upgrade storage controller ScotGRID delivery of Kit: Dec 2001 Group disk (re)organisation to match project Glasgow: MC-FARM Edinburgh: Datestore Proposal JREI: 2000

GridPP6 Collaboration MeetingScotGRID Report ScotGRID 1 st Year Review 9:45 Arrive - Coffee 10:00-10:15 Welcome (Freddie Moran) 10:15-10:35 ScotGrid Introduction (Tony Doyle) 10:35-10:50 Technical Status Overview (Akram Khan) 10:50-11:05 Cluster Operations (David Martin) 11:05-11:30 Coffee 11:30-11:50 ScotGrid Upgrade Plans (Steve Playfer) 11:50-13:00 IBM IT Briefing Discussion 13:00-14:00 Lunch 14:00-14:30 IBM IT Briefing Discussion 14:40-14:55 Grid Data Management - simulations (David Cameron) 15:10-15:30 Tea Particle Physics Applications 15:30-15:45 ATLAS (John Kennedy) 15:45-16:00 LHCb (Akram Khan) 16:00-16:15 BABAR (Steve Playfer) 16:15-16:30 CDF (Rick St Denis) ScotGrid Meeting at IBM Briefing Centre (Greenock) Friday 10th Jan ScotGrid Meeting at IBM Briefing Centre (Greenock) Friday 10th Jan Complete Success as you will see!

GridPP6 Collaboration MeetingScotGRID Report ScotGRID Statistics The amount of storage space in ScotGRID:used by each group Edinburgh (5TBytes) Glasgow (600 Gbytes)

GridPP6 Collaboration MeetingScotGRID Report ScotGRID:CPU Usage 24/6/02 – 6/1/2003 The % use by each group over the pervious weeks  startup phase  Christmas period  different applications

GridPP6 Collaboration MeetingScotGRID Report Forward Look: Introduction ScotGrid JREI project includes a mid- term hardware upgrade. As part of GridPP planning, we need to upgrade from Prototype to Production Tier 2 status by JREI funding left to be spent by June 2003: è Edinburgh £220k è Glasgow £30k £250k

GridPP6 Collaboration MeetingScotGRID Report Forward Look: Possible Upgrade Plan? Edinburgh kit  Glasgow Dual FastT TB xSeries 440 8* Xeon (1.9GHz) Scalable configuration

GridPP6 Collaboration MeetingScotGRID Report Forward Look: Front-End Grid Servers  Front end for EDG style Compute Engine/LCFG  Front end for EDG style Storage Engine  Overall ScotGrid Front end to arbitrate Grid services being requested? Would like to install Grid software on dedicated (modest-sized) servers. Decouples Grid software from Compute and Storage hardware. Will there be a standard configuration for Grid access to Tier 2 sites? (RLS/SlashGrid)

GridPP6 Collaboration MeetingScotGRID Report Towards a Production Tier-2 & beyond Q1 Q2 Q3 Q Q4 ProductionProduction Delivery of more Kit… End of JREI funding Start of ScotGRID-II Start of GridPP-II Links to other applications … Production Tier-2 Site Future Upgrades ?

GridPP6 Collaboration MeetingScotGRID Report Technical Support Group Core members of the group & invited to discuss wider issues: CORE: Akram Khan (Chair: Edinburgh) David Martin (sysadm: Glasgow) Roy de Ruiter-Koelemeiger (sysadm: Edinburgh) Gavin McCance (EDG: Glasgow) RA post (EDG: Edinburgh) INVITED: Paul Mitchell (sysadm: Edinburgh) Alan J. Flavell (Networking: Glasgow) Steve Traylen (EDG: RAL) IBM Team CORE: Akram Khan (Chair: Edinburgh) David Martin (sysadm: Glasgow) Roy de Ruiter-Koelemeiger (sysadm: Edinburgh) Gavin McCance (EDG: Glasgow) RA post (EDG: Edinburgh) INVITED: Paul Mitchell (sysadm: Edinburgh) Alan J. Flavell (Networking: Glasgow) Steve Traylen (EDG: RAL) IBM Team Webpage “technical group” of Support is a real issue we are just about ok but for a production Tier-2? Support is a real issue we are just about ok but for a production Tier-2?

GridPP6 Collaboration MeetingScotGRID Report All University traffic Packet filtering 1 Gb/s 2.5 Gb/s GlasgowEdinburgh ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms 4. glasgow-bar.ja.net ( ) ms ms ms 5. po9-0.glas-scr.ja.net ( ) ms ms ms 6. po3-0.edin-scr.ja.net ( ) ms ms ms 7. po0-0.edinburgh-bar.ja.net ( ) ms ms ms ( ) ms ms ms 9. vlan686.kb5-msfc.net.ed.ac.uk ( ) ms ms ms ( ) ms ms ms Traceroute

GridPP6 Collaboration MeetingScotGRID Report EDG Middleware: Replica Optimiser Simulation Using ScotGrid for large-scale simulation runs. è uses ~15MB memory for ~60 threads. è 2-12 hours/simulati on Results to appear in IJHPCA 2003.

GridPP6 Collaboration MeetingScotGRID Report BaBar: Monte Carlo Production (SP4) ScotGrid (= edin) 8 Million Events in 3 weeks ScotGrid (= edin) 8 Million Events in 3 weeks Expect to import some streams/skims to Edinburgh in 2003 After the upgrade to ~30TB there may be interest in using ScotGrid to add to the storage available at the RAL Tier A site

GridPP6 Collaboration MeetingScotGRID Report è CERN (932 k) and Bologna (857 k) è RAL (471 k) è Imperial College and Karlsruhe (437 k) è Lyon (202 k) è ScotGrid (194 k) è Cambridge (100 k) è Bristol (92 k) è Moscow (87 k) è Liverpool (70 k) è Barcelona (56 k) è Rio (32 k) è CESGA (28 k) è Oxford (25 k) LHCb: Production Centres We can be confident for the TDR production and in 56 days with the current configuration we can produce 10 Million events (March-April 2003). Included in Draft of LHCC document: B0->J/phi K0s

GridPP6 Collaboration MeetingScotGRID Report Summary and Outlook Exciting time for ScotGRID: Exciting time for ScotGRID: There has been a lot of effort during the past year to get ScotGRID up and operational – we have learnt many ticks! Operational Prototype Centre:  We have an operational centre  Meeting the short term needs of the applications with modest resources (HEP + Middleware + non-PP)  Proof of Principle for Tier-2 Operation (pre-grid) There is a lot that needs still to the done:  having a full production system (24*7) (opt-grid)  to prototype various architectural solutions for Tier-2  look towards upgrades with a view for LHC timetable Support & Resources are a real issue for the near term future (Q1-2004)

GridPP6 Collaboration MeetingScotGRID Report RLS Architecture Local Replica Catalogues LRC on Storage Element LRC on Storage Element LRC on Storage Element RLI LRC on Storage Element Multiply indexed LRC for higher availability RLI indexing over the full namespace (all LRCs are indexed) RLI indexing over a subset of LRCs LRC indexed by only one RLI Replica Location Indices GlasgowEdinburgh CERN A Replica Location Service (RLS) is system that maintains and provides access to information about the physical location of copies of data items. A Replica Location Service (RLS) is system that maintains and provides access to information about the physical location of copies of data items. Gavin McCance Alasdair Earl Akram Khan (starting Feb)