1. GridPP Project Overview
David Britton, Imperial College

2. The Challenge of the LHC
All interactions
9 orders of magnitude
The HIGGS

3. Tier Structure HPSS Tier 0 Tier 1 Tier 2 Tier 3 Tier 4 Online System
Tier2 Centre ~1 TIPS
Online System
Offline Farm ~20 TIPS
CERN Computer Centre >20 TIPS
RAL Regional Centre
US Regional Centre
French Regional Centre
Italian Regional Centre
Institute
Institute ~0.25TIPS
Workstations
~100 MBytes/sec
Mbits/sec
HPSS
One bunch crossing per 25 ns
100 triggers per second
Each event is ~1 MByte
Physicists work on analysis “channels”
Each institute has ~10 physicists working on one or more channels
Data for these channels should be cached by the institute server
Physics data cache
~PBytes/sec
~ Gbits/sec or Air Freight
~Gbits/sec
Tier 0
Tier 1
Tier 3
Tier 4
ScotGRID++ ~1 TIPS
Tier 2

4. Detector digitisation
Data Hierarchy
RAW
Recorded by DAQ
Triggered events
~2 MB/event
Detector digitisation
ESD
Pseudo-physical information:
Clusters, track candidates
(electrons, muons), etc.
Reconstructed
information
~100 kB/event
AOD
Physical information:
Transverse momentum,
Association of particles, jets,
(best) id of particles,
Physical info for relevant “objects”
Selected
information
~10 kB/event
TAG
Analysis
information
~1 kB/event
Relevant information
for fast event selection

5. GridPP Project Overview

6. “The Project Map”

7. LHC Computing Grid (LCG)
Help create LCG project
Components shared by LHC experiments; e.g. POOL
Prototyping the CERN Tier0/1 center. (Scalability issue)
Requirements; Monitoring; recommendations.
Deploying and operating the LHC Computing Grid
6.2*
39.2
25.0
15.3
5.3
TOTAL 91.1
GridPP 24.0
* LCG management
FTE in 2002

8. LHC Computing Grid
LCG-1
Release

9. LCG Status Certification and distribution process established
Middleware package – components from –
European DataGrid (EDG)
US (Globus, Condor, PPDG, GriPhyN)  the Virtual Data Toolkit
Agreement reached on principles for registration and security
RAL to provide the initial grid operations centre
FZK to operate the call centre
Initial service being deployed now to 10 centres US, Europe, Asia
Expand to other centres as soon as the service is stable
On the map, the red dots are the first wave site, the blue dots are the sites that say they will be ready to come on in the next 6-12 months.
Status on 24 July 2003 – I will an update on the 11th August
There are two activities going on in parallel:
-- Pre-production software is being deployed to 10 Regional Centres (Academia Sinica Taipei, BNL, CERN, CNAF, FNAL, FZK, IN2P3 Lyon, Moscow State Univ., RAL, Univ. Tokyo). This is a final validation of the distribution and installation process and tools, and establishes the network of service support people, but the Grid middleware is not the final version yet. About half of the sites have successfully installed this – a few (including BNL and Lyon) are very slow.
-- Final testing is under way of the first production set of Grid middleware. This is being done by a group of grid experts from the LHC experiments (the “loose cannons”). There are a few “show stoppers” and about 15 very serious bugs.
The target is to install the production software by the end of the month – one week from now!
Academia Sinica Taipei, BNL, CERN, CNAF, FNAL, FZK, IN2P3 Lyon, Moscow State Univ., RAL, Univ. Tokyo

10. European DataGrid (EDG)
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.
EDG 2.0
EDG 2.1

11. European DataGrid UK Roles Quality Assurance Representative 0.5
Deputy Leader, (Iteam)
2.0
Group Leader, Deputy Leader, Iteam, ATF, QA EU EU
CA Group Manager
3.0
Deputy Leader, Security Group Leader, ATF
3.7*
* Includes Security
Group Leader
4.0
FTE
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.

12. Job Checkpointing in EDG2.0
European DataGrid – WP1
WP1 Workload Management
Job Checkpointing in EDG2.0
Job checkpoint states
saved in the LB server
Retrieval
of job
checkpoint
Logging &
Bookkeeping
Server
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.
Job
Also used (even in rel. 1) as repository of job status info
Already proved to be robust and reliable
The load can be distributed between multiple
LB servers, to address scalability problems
Saving of
job checkpoint state
state.saveState()

13. See OptorSim Demo at PPARC Stand
European DataGrid – WP2
WP2 Data Management
UK Contributions
RM in EDG2.0
User Interface or
Worker Node
Resource Broker
Virtual Organization
Membership Service
Information Service
Replica Metadata
Catalog
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.
Replica Location
Service
Replica Manager
See OptorSim Demo at PPARC Stand
Replica Optimization
Service
Storage
Element
Storage
Element SE
Monitor
Network Monitor

14. European DataGrid – WP3 RGMA in EDG2.0 UK Product
WP3 Information and Monitoring Services
RGMA in EDG2.0
R-GMA Consumers
LDAP
InfoProvider
GIN
Server
Stream Producer
Consumer (CE)
Consumer (SE)
Consumer (SiteInfo)
RDBMS
Latest
Producer
GOUT
ConsumerAPI
Archiver (LatestProducer)
R-GMA
GLUE
Schema
Push mode
Updates every 30s
>70 sites (simul.)
UK Product
RGMA talk Thursday 10.15am
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.

15. European DataGrid – WP4 UK Product WP4 Fabric Management
LCFG configuration software from Univ. Edinburgh was used from Month 12 onwards of the EDG project.
Newer version, LCFGng, in EDG-2.0 and in LCG-1
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.

16. European DataGrid – WP5 SE in EDG2.0 UK Product
WP5 Mass Storage Management
SE in EDG2.0
Client App
API
SE HTTP
library
SSL socket
AXIS
SE core SE
Java Client
Tomcat
The design of the SE follows a layered model with a central core handling all paths between client and MSS.
Core is flexible and extensible making it easy to support new protocols, features and MSS
Java
Client API
C Client
RMANMAN
Apache
Web Service
UK Product
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.

17. Testbed successfully demonstrated during
European DataGrid – WP6
WP6 Testbed
EDG Application testbed:
More than 40 sites
More than 1000 CPUs
5 Terabyte of storage
Testbed successfully demonstrated during
2nd EU review in Feb 2003
Large UK participation
DataGrid passed the 2nd EU review in Feb. The reviewers congratulated the EU management for taking the risk of concentrating on Quality but hope that there will be no lost functionality by the end of the project.
As GridPP and particularly GridPP2 progress this is a lesson that we need to bare in mind; one can’t be too ambitious if we want to establish a production quality grid.
At this stage the status is EDG 2.0 has been released and now intense work on integrating and hardening.

18. (Wed 12:10: High speed data transfers)
European DataGrid – WP7
(Wed 12:10: High speed data transfers)
WP7 Network Services

19. Applications

20. Multilayered Grid architecture
Applications: GANGA
Server
Bookkeeping DB
Production
EDG UI
PYTHON SW BUS
XML RPC server
XML RPC module
GANGA Module
OS Module
Athena\GAUDI
GaudiPython
PythonROOT
GUI
Job
Configuration
Remote user
(client)
Local Job
LAN/WAN
GRID
LRMS
Underlying GRID services (GLOBUS toolkit)
GRID middleware (EDG, PPDG,…)
Application specific layer (Athena/Gaudi, …)
GUI interface
OS and Network services
Multilayered Grid architecture
GANGA
A common interface to the Grid for Atlas and LHCb

21. See GUIDO demo at the PPARC booth
Applications: GUIDO
See GUIDO demo at the PPARC booth

22. Applications: CMS
Adding RGMA to BOSS

23. Applications: CDF/D0

24. Applications: CDF/D0 D0 plan to reprocess 22 TB of DST using
SAMGrid starting mid September
See SAMGrid Demo at PPARC Stand

25. Tier-1/A

26. Tier-2 ScotGrid … functional Tier-2 Four consortia established.
½ post funded at each.
A lot of hardware exists.

28. Testbed

29. All testbed sites can be said to be truly on a Grid by virtue of their registration in a comprehensive resource information publication scheme, their accessibility via a set of globally enabled resource brokers, and the use of one of the first scalable mechanisms to support distributed virtual communities (VOMS). There are few such Grids in operation in the World today.

30. Data Challenges

31. ATLAS DC1 Phase 2
UK largest producer

32. Interoperability,Dissemination, Resources

33. The Future GridPP2 proposal: http:www/gridpp.ac.uk/docs/gridpp2/
“From Testbed to Production”

34. Summary Achievements: Status:
A major factor in establishing the LCG project
Leadership roles within 5 of the 8 (applicable) EDG workpackages.
Significant middleware development.
Deployed the UK’s largest testbed (16 sites and >100 servers).
Integration with EU-wide programme and linked to Worldwide efforts.
Active Tier-1/A production centre, meeting demands.
Tier-2 resources identified and a structure being developed.
Established productive links with the experiment applications.
Developed a proposal for a follow-on project, GridPP2.
Responded to the “LHC Computing Challenge”: Defined and initiated a programme that will be the basis for LHC computing.
Status:
At the midpoint of GridPP1, 95 out of 182 tasks (52%) have been
successfully completed.
The LCG-1 release is imminent; a landmark moment.