1. Tony Doyle
GridPP Collaboration Meeting
9 May 2002

2. Tony Doyle - University of Glasgow
Outline
The Vision Thing…
Grid
Scale
Integration
Dissemination
LHC Analyses
Other Analyses
DataGrid
LCG
Interoperability
Infrastructure
Finanaces
Summary
Tony Doyle - University of Glasgow

3. Tony Doyle - University of Glasgow
Inspiration??
Tony Doyle - University of Glasgow

4. Tony Doyle - University of Glasgow
GridPP Documents
Tony Doyle - University of Glasgow

5. Tony Doyle - University of Glasgow
GridPP Vision
From Web to Grid - Building the next IT Revolution
Premise
The next IT revolution will be the Grid. The Grid is a practical solution to the data-intensive problems that must be overcome if the computing needs of many scientific communities and industry are to be fulfilled over the next decade.
Aim
The GridPP Collaboration aims to develop and deploy the largest-scale science Grid in the UK for use by the worldwide particle physics community.
Many Challenges..
Shared distributed
infrastructure
For all experiments
Tony Doyle - University of Glasgow

6. Tony Doyle - University of Glasgow
GridPP Objectives
1. SCALE: GridPP will deliver the Grid software (middleware) and hardware infrastructure to enable the testing of a prototype of the Grid for the LHC of significant scale.
2. INTEGRATION: The GridPP project is designed to integrate with the existing Particle Physics programme within the UK, thus enabling early deployment and full testing of Grid technology and efficient use of limited resources.
3. DISSEMINATION: The project will disseminate the GridPP deliverables in the multi-disciplinary e-science environment and will seek to build collaborations with emerging non-PPARC Grid activities both nationally and internationally.
4. UK PHYSICS ANALYSES (LHC): The main aim is to provide a computing environment for the UK Particle Physics Community capable of meeting the challenges posed by the unprecedented data requirements of the LHC experiments.
5. UK PHYSICS ANALYSES (OTHER): The process of creating and testing the computing environment for the LHC will naturally provide for the needs of the current generation of highly data intensive Particle Physics experiments: these will provide a live test environment for GridPP research and development.
6. DATAGRID: Grid technology is the framework used to develop this capability: key components will be developed as part of the EU DataGrid project and elsewhere.
7. LCG: The collaboration builds on the strong computing traditions of the UK at CERN. The CERN working groups will make a major contribution to the LCG research and development programme.
8. INTEROPERABILITY: The proposal is also integrated with developments from elsewhere in order to ensure the development of a common set of principles, protocols and standards that can support a wide range of applications.
9. INFRASTRUCTURE: Provision is made for facilities at CERN (Tier-0), RAL (Tier-1) and use of up to four Regional Centres (Tier-2).
10. OTHER FUNDING: These centres will provide a focus for dissemination to the academic and commercial sector and are expected to attract funds from elsewhere such that the full programme can be realised.
(…. WHAT WE SAID WE COULD DO IN THE PROPOSAL)
Tony Doyle - University of Glasgow

7. Grid – A Single Resource
Glenn Patrick
Many millions
of events
Various conditions
Many samples
GRID
A unified approach
Peta Bytes of data storage
Distributed resources
Many 1000s of computers required
Worldwide collaboration
Heterogeneous operating systems
Tony Doyle - University of Glasgow

8. Grid - What’s been happening?
A unified approach
OGSA
A lot…
GGF4, OGSA and support of IBM (and others)
[as opposed to .NET development framework and passports to access services]
Timescale? September 2002
W3C architecture for web services
Chose (gzipped) XML as opposed to other solutions for metadata descriptions… and web-based interfaces
linux
[as opposed to other platforms… lindows??]
C++ (experiments) and C, Java (middleware) APIs
[mono - Open Source implementation of the .NET Development Framework??]
Tony Doyle - University of Glasgow

9. GridPP Context (Internally)
Provide architecture and middleware
Future LHC Experiments
Running US Experiments
Build Tier-A/prototype Tier-1 and Tier-2 centres in the UK and join worldwide effort to develop middleware for the experiments
Use the Grid with simulated data
Use the Grid with real data
Tony Doyle - University of Glasgow

10. Using Grid Middleware and DataGrid Tools
A unified approach
1. Authentication
grid-proxy-init
2. Job submission to DataGrid
dg-job-submit
3. Monitoring and control
dg-job-status
dg-job-cancel
dg-job-get-output
4. Data publication and replication
globus-url-copy, GDMP
5. Resource scheduling – use of CERN MSS
JDL, sandboxes, storage elements
Important to
implement this
for all experiments…
Tony Doyle - University of Glasgow

11. Tony Doyle - University of Glasgow
EDG TestBed 1 Status
GRID
A unified approach
Web interface showing status of (~400) servers at testbed 1 sites
GRID
extend to all expts
Tony Doyle - University of Glasgow

12. LHC computing at a glance
1. scale
The investment in LHC computing will be massive
LHC Review estimated 240MCHF (before LHC delay)
80MCHF/y afterwards
These facilities will be distributed
Political as well as sociological and practical reasons
Europe:
267 institutes, 4603 users
Elsewhere: 208 institutes, 1632 users
Tony Doyle - University of Glasgow

13. GridPP funded Staff at CERN
7. LCG
IT/ADC
Ian Neilson from 16th April. Working on certificate authority then scaling up EDG testbed.
IT/API
Jacek Generowicz from 1st Feb. Working on GEANT4, in particular, tracking & visualisation
Maria Girone from 1st April. Settling in.
IT/FIO
John Hearns from 1st April. Configuration & installation management; relates to WP4 effort and LCFG.
Bill Tomlin from 16th April. State Management.
Little acorns..
Tony Doyle - University of Glasgow

14. Tony Doyle - University of Glasgow
RTAG Status
Tony Cass
6 RTAGs created to date:
RTAG1 (Persistency Framework; status: completed)
RTAG2 (Managing LCG Software; status: running)
RTAG3 (Math Library Review; status: running)
RTAG4 (GRID Use Cases; status: starting)
RTAG5 (Mass Storage; status: running)
RTAG6 (Regional Centres; status: starting)
Two more in advanced state of preparation:
Simulation components
Data Definition Tools
Tony Doyle - University of Glasgow

15. Applications area has momentum
Tony Cass
Torre Wenaus started as Area Manager in March.
Activities in the Applications area are now well organised. See
Weekly Applications area meetings initiated together with “Architects Forum”.
Active work for Software Process and Persistency. Work plan must be developed for SC2 approval.
Tony Doyle - University of Glasgow

16. Fabrics & Grid Deployment
Tony Cass
LCG Level 1 Milestone: deploy a Global Grid Service within 1 year
sustained 24 X 7 service
including sites from three continents
identical or compatible Grid middleware and infrastructure
several times the capacity of the CERN facility
and as easy to use
Ongoing work at CERN to increase automation and streamline configuration, especially for migration to RedHat 7.2.
Aim to phase out old CERN solutions by mid-2003.
Tony Doyle - University of Glasgow

17. Tony Doyle - University of Glasgow
LCG Timeline
1. timescale
Prototype of Hybrid Event Store (Persistency Framework)
Hybrid Event Store available for general users
applications
Distributed production using grid services
Full Persistency Framework
Distributed end-user interactive analysis
2002
2005
2004
2003
Q1 Q2 Q3 Q4
LHC Global Grid TDR
grid
“50% prototype” (LCG-3) available
LCG-1 reliability and performance targets
First Global Grid Service (LCG-1) available
Tony Doyle - University of Glasgow

18. Tony Doyle - University of Glasgow
Be a part of this?
Notes
1. The minimum period for LTA is 3 months. It is expected that a work programme will be typically for 6 months (or more).
2. Prior DataGrid and LHC (or other) experiments' Grid work are normally expected.
3. It is worthwhile reading
in order to get an idea of the areas covered, and the emphasis placed, by the LCG project on specific areas (building upon DataGrid and LHC experiments' developments).
4. Please send all enquiries and proposals to:
Tony Doyle and
Tony CASS
LCG Development
– Long Term Attachment at CERN
This will enable Grid developments in the UK to be (more) fully integrated with long-term Grid development plans at CERN.
The proposed mechanism is:
1. submit a short one-page outline of current and proposed work, noting how this work can best be developed within a named team at CERN, by to the GridPP Project Leader (Tony Doyle) and GridPP CERN Liaison (Tony Cass).
2. This case will be discussed at the following weekly GridPP PMB meeting and outcomes will be communicated as soon as possible by following that meeting.
Tony Doyle - University of Glasgow

19. Summary of LCG presentation
Tony Cass
Project got under way early this year
Launch workshop and early RTAGs give good input for high-level planning …
… to be presented to LHCC in July
New plan takes account of first beam in 2007
No serious problems foreseen in synchronising LCG plans with those of the experiments
Collaboration with the many Grid projects needs more work
Technical collaboration with the Regional Centres has to be established
Recruitment of special staff going well (but need to keep the recruitment momentum going)
Very serious problem with materials funding
Tony Doyle - University of Glasgow

20. Tony Doyle - University of Glasgow
Building upon Success
6. DataGrid
The most important criterion for establishing the status of this project was the European Commission review on March 1st 2002.
The review report of project IST DATAGRID is available from PPARC.
The covering letter states “As a general conclusion, the reviewers found that the overall performance of the project is good and in some areas beyond expectations.”
The reviewers state “The deliverables due for the first review were in general of excellent quality, and all of them were available on time… All deliverables are approved. The project is doing well, exceeding expectations in some areas, and coping successfully with the challenges due to its size.”
Tony Doyle - University of Glasgow

21. Tony Doyle - University of Glasgow
6. DataGrid
Tony Doyle - University of Glasgow

22. Identifiable Progress...
3. Dissemination t0 t1
Tony Doyle - University of Glasgow

23. UK TierA/prototype Tier-1 Centre
9. Infrastructure
                       
Roles
Tier-A Centre for BaBar
EDG testbed(s)
LCG prototype Tier-1 Centre
prototype Tier-1 for LHC experiments (Data Challenges independent of LCG development…)
Interworking with other UK resources (JIF, JREI, eSC) = UK portal
existing LEP, DESY and non-accelerator experiments
Purchases
First year = Hardware Advisory Group (HAG1)
Balance between cpu, disk, and tape
Experts on specific technologies
Propose more HAGs (2 and 3)..
Needs to be successful in all roles Report progress/problems
Tony Doyle - University of Glasgow

24. Rollout of the UK Grid for PP
9. Infrastructure
Operational stability of GridPP middleware = Testbed team
“the gang of four” … Andrew McNab, Steve Traylen, Dave Colling (other half) and Owen Moroney
Ensures the release of “Testbed” quality EDG software
documentation
lead for other system managers in terms of implementation
pre-defined software cycle releases (2 months..)
Subject of the Rollout Plan… “Planning for EDG Testbed software deployment and support at participating UK sites” (Pete Clarke, John Gordon)
See Andrew and John’s talks..
LCG is the proposed mechanism by which the EDG testbed at CERN becomes an LCG Grid Service. The evolution of the EDG testbed to the LCG Grid Service will take account of both EDG and US grid technology. Need to take account of this..
Tony Doyle - University of Glasgow

25. Tony Doyle - University of Glasgow
Longer Term..
9. Infrastructure
LCG Grid Service
Takes account of EDG and US grid technology
A large-scale Grid resource, consistent with the LCG timeline, within the UK.
Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUs = GridPP in Sept 2004
“50% prototype”?
Tony Doyle - University of Glasgow

26. GridPP Context (Externally)
Neil Geddes
9. Interoperability
Tony Doyle - University of Glasgow

27. Tony Doyle - University of Glasgow
GLUE
9. Interoperability
How do we integrate with developments from elsewhere in order to ensure the development of a common set of principles, protocols and standards that can support a wide range of applications?
GGF…
Within the Particle Physics community, these ideas are currently encapsulated(??) in the Grid Laboratory Uniform Environment (GLUE).
Recommend this as a starting point for the wider deployment of Grids across the Atlantic. See (Ruth Pordes et al.)
Tony Doyle - University of Glasgow

28. Tony Doyle - University of Glasgow
9. Interoperability
Tony Doyle - University of Glasgow

29. Tony Doyle - University of Glasgow
9. Interoperability
= Minimal
e-Bureaucracy
Tony Doyle - University of Glasgow

30. Tony Doyle - University of Glasgow
£17m 3-Year Project
Dave Britton
10. Finances
Five components
Tier-1/A = Hardware + ITD Support Staff
DataGrid = DataGrid Posts + PPD Staff
Applications = Experiments Posts
Operations = Travel + Management + e Early Investment
CERN = LCG posts + Tier-0 + e LTA
Tony Doyle - University of Glasgow

31. 0. Projected Staff Effort [SY]
Status at GridPP3
Area GridPP @CERN CS
WP1 Workload Management [IC] [IC]
WP2 Data Management [Ggo] [Oxf] WP12
WP3 Monitoring Services [RAL, QMW] 1.0 [HW]
Security [RAL] [Oxf]
WP4 Fabric Management [Edin., L’pool]
WP5 Mass Storage [RAL, L’pool]
WP6 Integration Testbed [RAL/M’cr/IC/Bristol]
WP7 Network Services [UCL/M’cr] 1.0 [UCL]
WP8 Applications
ATLAS/LHCb (Gaudi/Athena) [Oxf, Cam, RHUL, B’ham, RAL]
CMS [IC, Bristol, Brunel]
CDF/D0 (SAM) [IC, Ggo, Oxf, Lanc]
BaBar [IC, M’cr, Bristol]
UKQCD [Edin.]
Total >
Tony Doyle - University of Glasgow

32. Tony Doyle - University of Glasgow
1. Recruitment
EDG Funded Posts (Middleware/Testbed)
All 5 in post + 1 additional
EDG Unfunded Posts (Middleware/Testbed)
15 out of 15 in post
GridPP Posts (Applications + Tier1/A)
Allocated Dec 2001
~14 out of 15 in post
CERN Posts
First Round = 105 Applicants, 12 Offers, 9 Accepted
4 in Applications, 2 Data Management, 3 Systems
Second Round = 140 applicants, 9 Offers
Third Round ~ 70 Applicants
Aim ~ 28 posts
Tony Doyle - University of Glasgow

33. 2. Monitoring Staff Effort [SM]
Robin Middleton
Tony Doyle - University of Glasgow

34. 3. Progress towards deliverables..
Pete Clarke
Tony Doyle - University of Glasgow

35. Tony Doyle - University of Glasgow
-1. Next steps..
10. Finances
O(100k)
e CLRC support through to Sept 04
Other experiments – unfunded in peer review process
Tier-2 centres – unfunded initially
£2.3m
eDIKT (e-Data, Information and Knowledge Transformation) [SHEFC] Particle Physics = application area - assignment of two (of twelve) FTEs in initial planning. Discussions ongoing with EPCC.
O(€100m)
The first call for Framework 6 will be early next year.
Call out now for expressions of interest for new networks and integrated projects.
Draft document led by David Williams (CERN) “Enabling Grids and e-Science in Europe” plans to extend the current paradigms with CERN at its focus as the European e-Science Centre.
We believe this is the right approach.
Incorporates the UK’s e-Science agenda, adding a European dimension. It also recognises the central role of CERN and builds upon the recent successes of EDG.
Contact: Neil Geddes
Tony Doyle - University of Glasgow

36. Tony Doyle - University of Glasgow
Summary
A vision is only useful if its shared
Grid success is fundamental for PP
Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUs? = GridPP in Sept 2004
Integration – ongoing..
Dissemination – external and internal
LHC Analyses – we’re getting there..
Other Analyses – ditto - look forward to Roger’s talk
DataGrid - - major investment = must be successful
LCG – Grid as a Service
Interoperability – sticky subject
Infrastructure – Tier-A/1 in place, Tier-2’s to follow…
Finances – (very well) under control
Next steps on framework 6..
CERN = EU’s e-science centre
Co-operation required with other disciplines
Monitoring mechanisms in place…
Emphasis on deliverables
End of “business” section
Tony Doyle - University of Glasgow

37. Tony Doyle - University of Glasgow
UML Diagram
GRID
A unified approach
True
Vision??
Tony Doyle - University of Glasgow

38. 4th Collab Meeting Agenda
Wish I’d b(r)ought a network card Welcome!
Tony Doyle - University of Glasgow