Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting GridPP2 Status Tony Doyle

OR who will win the World Cup?

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting World Cup Performance? France Italy Sweden -v- Germany

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Outline 1.90% reliable World Grid? 2.gLite Medium-term resource planning 4.performance improvements 5.Dissemination and what to say to taxi drivers 6.The GridPP2 Project is halfway through: how many targets have been met? 7.EGEE phase I – industrial liaison as part of the next phase 8.Worldwide LCG (Memorandum of) Understanding 9.File transfers.. 10.Know your users World cup prediction (based on sound metrics?)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting All key objectives have been reached for the end of 2005 and installation is now proceeding smoothly. Three quarters of the machine has been liberated for magnet installation and interconnect work is proceeding in 2 octants in parallel. Magnet installation is now steady at 25/wk. Installation will finish end March The machine will be closed in August Every effort is being made to establish colliding beams before the end of 2007 at reduced energy. The full commissioning up to 7 TeV will be done during the winter shutdown ready for a Physics run at full energy in LHC? Status of the LHC Project Lyndon Evans Machine Advisory Committee 15 June 2006

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting The Service Challenge programme this year must show that we can run reliable services Grid reliability is the product of many components – middleware, grid operations, computer centres, …. Target for September –90% site availability –90% user job success Requires a major effort by everyone to monitor, measure, debug First data will arrive next year NOT an option to get things going later Too modest? Too ambitious? Challenges for 100 Computing Centres in 20 Countries Les Robertson HEPiX Meeting, Rome, 5 April 2006 WLCG?

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting gLite 3.0 What is gLite-3.0? LCG-2.7 and updates gLite WMS/LB gLite CE gLite/LCG WN gLite/LCG UI FTS (Service) FTA (Agents)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting OC Actions 1.GridPP TO PROVIDE DATA ON WHAT FRACTION OF THE REGISTERED USERS WERE MAKING THE GREATEST USAGE OF THE RESOURCES. ONGOING 2.GridPP TO PROVIDE PPARC WITH A TIER 1 PURCHASE PLAN FOR FY06. DONE 3.GridPP to provide data on experiments’ increased usage of Tier 2 resources. DONE 4.GridPP to provide an update of the performance metrics. DONE 5.GridPP to present a draft GridPP3 proposal to the next meeting. DONE (version 0.6) 6.GridPP to circulate procedures adopted by Grid Security Vulnerability Group to Committee members. DONE 7.GridPP to provide a paper to the next meeting justifying the proposed Tier 1 hardware spend in FY07 against other spending options. ONGOING 8.GridPP to describe its relationship with the e-Science Core Programme more fully. ONGOING 9.GridPP to provide PPARC, on a post-by-post basis, details of the cost of extending posts finishing before new funding is expected to be in place (end of March 2008). DONE

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting WLCG MoU 17 March 2006 PPARC signed the Memorandum of Understanding with CERN Commitment to UK Tier-1 at RAL and the four UK Tier-2s to provide services and resources. Will need to propagate through LFRC..

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting UK pledges & medium- term planning RAL, UK Pledged Planned to be pledged CPU (kSI2K) Disk (Tbytes) Tape (Tbytes) As defined in summer Tier-1 (v26b) plan 2007 or Tier-2 GridPP MoU, followed by pessimistic guess 2.August 2005 “minimal Grid” 3.GridPP3 proposal (see Dave’s talk) Need to update 2007 pledges by Sept. 06 UK, Sum of all Federations Pledged Planned to be pledged CPU (kSI2K) Disk (Tbytes)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Capacity Planning Considerations… The Tier-1/A capacity originally planned to be available for 06Q1 was put into production in late April 2006 (CPU). The disk capacity is scheduled to be available in early August. The new SL8500 tape robot began providing a production service in March. The first three T10K tape drives for the GridPP tape service are expected to be delivered in July together with 200TB of tape media. The SL8500 robot will be upgraded from 6000 to slots (paid for by CCLRC). Tenders for 500 kSI2k and 237 TB of disk at the end of June. Good progress is being made on the deployment of CASTOR2 (providing HSM capability and SRM interface to storage) which remains on schedule for a production service in September. For the Tier-2 centres, additional capacity was made available in 06Q1, with the incorporation of capacity at two additional large centres (Manchester and Liverpool). The available CPU in the first quarter increased to 3703 kSI2k such that 75% of the MoU commitment has now been met, with disk increasing to 263 TB. CPU utilisation of this much larger resource was 23% with overall disk utilisation improved at 61% in 06Q1. Additional capacity improvements are envisaged at Bristol, Cambridge, Glasgow and QMUL during this year. The GridPP resource utilisation outturn for 2005 updated to include 06Q1 is available from

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Dissemination If a taxi driver asks you what you do.. Mention the Grid by numbers Or the BBC.... and avian flu?

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Project Status To be reviewed at Friday’s OC.. Good progress, according to plan Glass half full....and half empty Metric OK Metric not OK Tasks Complete Tasks Overdue Tasks due in next 60 days Items Inactive Tasks not Due Change Forms 88 (91%) 9127 (49%)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Performance.. The Grid isn’t a swordfish (or a barracuda..) It’s a shoal of large and small goldfish

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Tier-0 to Tier-1 worldwide data transfers > 950MB/s for 1 week peak transfer rate from CERN of >1.6GB/s Need high data rate transfers to/from CERN as a routine activity

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Tier-1 to Tier-2 UK data transfers >1000Mb/s for 3 days peak transfer rate from RAL of >1.5Gb/s Need high data rate transfers to/from RAL as a routine activity (see later talks)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Tier-1 RAL Tier-1 Tier-2 NorthGrid Experiment computing models define actual data flows Need to test these flows over the summer..

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Moving Files… keeping track Volatile –Temporary and sharable copy of an MSS resident file –If not pinned it can be removed by the garbage collector as space is needed (typically according to LRU policy) Durable –File can only be removed if the system has copied it to an archive Permanent –System cannot remove file Users can always explicitly delete files The experiments only want to store files as permanent –Even scratch files  will be explicitly removed by experiment

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Sustained Data Rates CERN  Tier-1s CentreALICEATLASCMSLHCbRate into T1 MB/sec (pp run) ASGC, TaipeiXX100 CNAF, ItalyXXXX200 PIC, SpainXXX100 IN2P3, LyonXXXX200 GridKA, GermanyXXXX200 RAL, UKXXX150 BNL, USAX200 FNAL, USAX200 TRIUMF, CanadaX50 NIKHEF/SARA, NLXXX150 Nordic Data Grid FacilityXX50 Totals1,600 Design target is twice these rates to enable catch-up after problems. Note this also for Tier-1 to Tier-2 rates. Not a problem?..

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting gLite 3.0 deployment Upgrades are supported from LCG Appears to work well Sites need to keep on the upgrade path A reasonably well-defined deployment release cycle Release cycle is getting (somewhat) shorter

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Know your users.. On the Grid you don’t But you do have some measures to guide you.. JRA2/QoS/JobsMetrics/JobMetrics.htm#DIS CLAIMER: JRA2/QoS/JobsMetrics/JobMetrics.htm#DIS CLAIMER Source of all knowledge (inc. World Cup predictions)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Active Users (All VOs)

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Job success? Overview

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Job Success by LHC experiment ALICE CMS ATLAS LHCb

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Active Users by LHC experiment ALICE (8) CMS (150) ATLAS (70) LHCb (40)

Enabling Grids for E-sciencE EGEE-II INFSO-RI Talk Title EGEE-I Review “The project was successfully completed and followed high standards with respect to project management, software development, integration and operations. The objective of a reliable production grid of significant size, which is professionally operated, monitored, maintained and continuously expanded, has been achieved. The average number of daily jobs and the number of involved sites are impressive. The EGEE brand was successfully introduced world-wide. The training efforts and achievements continued to be impressive. The successful merge of LCG and gLite middleware distributions provides a solid foundation for the future evolution of the EGEE middleware. The project management structure very well adapted to the evolving requirements of the project. EGEE successfully fulfilled its role as an incubator and as a driver for linking European grid projects to world-wide grid activities. The stronger involvement in international standardization efforts is well recognized, although the impact could have been stronger and more visible. For instance, the VO Management Service (VOMS) is becoming a de facto standard in many major research grids world-wide, but EGEE's contributions are not sufficiently well known. All deliverables are of high quality and more appropriately sized and focused than in the two previous reviews. All deliverables are accepted. There were no notable deviations from the work plan. Resources and major costs were necessary and of reasonable economy.” EGEE-I worked at many levels

Enabling Grids for E-sciencE EGEE-II INFSO-RI Talk Title Grids and Business – key points Trust - Not use to sharing resources Security - Sensitive data with sensitive applications Business models – what can be charged for as a service Guaranteed QoS – Service Level Agreements Accounting - tracking resources usage in multi-admin context Standards – to encourage long-term investment Applications – need to support legacy applications Portability – across multiple platforms and implementations Open source support – robust reference implementation Software license management – how to generate revenue in a grid context EGEE-II provides an excellent framework for collaborating with business on these subjects

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Where are we now? Top 10 Since the last collaboration meeting a lot has happened.. 1.Progress has been made in the release of gLite-3.0 (first middleware fully integrating all EGEE and LCG components) 2.gLite-3.0 efficiently deployed at 11 UK sites 3.Tier-2 resources on the Production Grid beginning to be fully utilised 4.Many measured performance improvements (see Jeremy’s talk) 5.The GridPP2 Project is halfway through: 49% of its targets met, 91% of the metrics within specification 6.EGEE phase I reviewed – commended by the EU 7.Dissemination: lead news on BBC technology web site, GridPP overview for MPs circulated. 8.In March 2006 PPARC signed the worldwide LCG MoU 9.Significant planning performed for GridPP3 (see Dave’s talk) 10.Work starting on large-scale experiment-specific file transfers and improving site performance..

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting Summary Wot no World Cup prediction?

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting How will England fare?

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting World Cup Predictions Result: –Switzerland0Ukraine3 –Stop Press: Swiss team have been asked by England for advice on penalty taking.. SwitzerlandUkraine

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting World Cup Predictions Result: –Holland0Portugal1 –88 –22 Need a new metric… HollandPortugal

Tony Doyle - University of Glasgow 27 June 2006Collaboration Meeting World Cup Prediction: The winner will be… the most remote site on the EGEE Grid?..Brazil