1. Tony Doyle
GridPP Collaboration Meeting
14 Feb 2002

2. Tony Doyle - University of Glasgow
GridPP Status Report
Starting Point
Project Management Structures
Experiments Board (EB)
CERN and the LHC Computing Grid (LCG)
Technical Board (TB)
Project Management Board
High Level Deliverables
Staff Effort and Recruitment
DataGrid
“Inter Grids”
2002 Challenges
UK Testbed = “The Grid”
Summary
Agenda
Tony Doyle - University of Glasgow

3. Tony Doyle - University of Glasgow
GridPP Management
Who’s who?..
c.f. Steve’s
web list of contacts
Tony Doyle - University of Glasgow

4. Tony Doyle - University of Glasgow
Collaboration Board
Chair Steve Lloyd
Birmingham Peter Watkins Liverpool Themis Bowcock
Bristol Greg Heath Manchester Robin Marshall
Brunel Peter Hobson Oxford Susan Cooper
CERN Les Robertson Queen Mary Tony Carter
Cambridge Janet Carter Royal Holloway John Strong
Durham Mike Whalley RAL Ken Peach
Edinburgh Steve Playfer Sheffield Neil Spooner
Glasgow David Saxon Sussex David Wark
IC Peter Dornan Swansea Chris Alton
Lancaster Peter Ratoff UCL Peter Clarke
Project Leader (in attendance) Tony Doyle
e-Science Director (in attendance) Neil Geddes
Members are 'Group Leaders' of all GridPP Institutes
Chair elected by Board
Tony Doyle - University of Glasgow

5. Tony Doyle - University of Glasgow
Experiments Board
ATLAS Roger Jones, Andy Parker
CMS Dave Britton, Dave Newbold
LHCb Nick Brook (Chair), Glenn Patrick
ALICE Ian Bloodworth
BaBar Roger Barlow (Deputy Chair)
D0 Iain Bertram
CDF Todd Huffman
Others Dan Tovey
Theory Alan Irving
CERN Jamie Shiers
TB Cross Members: Pete Clarke, tbd
Members chosen by Experiments
Chair and Deputy elected by Board
Tony Doyle - University of Glasgow

6. Recruitment Timetable
8th October - EB input deadline
Inputs:
1. TECHNICAL CASE Complete
2. ASSESMENT OF HOW MUCH EFFORT IS REQUIRED Complete
3. WHO WILL CONTRIBUTE TO/LEAD THESE DEVELOPMENTS Complete
4. WHAT IS REMAINING TO BE DONE - I.E. A POSTS REQUIREMENT Complete
5. A JOB SPECIFICATION OF THE OUTSTANDING POST(S) INCLUDING DELIVERABLES AND MILESTONES Complete
19th October - TAG assessment document
Next step - technical deployment issues
26th October - PMB definition of tasks
Next step - deployment/resource issues
first review of CERN posts
2nd November - Tender document
5-6th November - Collaboration Meeting
30th November- Deadline for submission by Institutes
7th December - PRSC mtg.
14th December- Oversight Committee mtg.
Ambitious timescales.
Experiments Input?
“good” to “very good”
Tony Doyle - University of Glasgow

7. Tony Doyle - University of Glasgow
Experiments Board
Nick Brook
New phase for EB – moving from focussing on post “creation” to the need for exploitation and utilisation of GridPP resources
Remit:
To ensure that the GridPP project is ‘experimentally-driven’
‘Long term’ planning – assessment of needs and reqts from GridPP
Dissemination of experiences and information between UK collaborations
To facilitate close working relationship between expts.
Overall liaison between the expts & GridPP
Tony Doyle - University of Glasgow

8. Tony Doyle - University of Glasgow
EB Meetings
Nick Brook
Next meeting
Late March, after LCG startup workshop
Areas of discussion:
status of recruitment
status & experience of Grid work
long term plans (changes to original submissions)
reporting procedures to GridPP
testbed experiences
management & "booking" of  UK Grid resources
quarterly reporting to TB
Tony Doyle - University of Glasgow

9. Tony Doyle - University of Glasgow
CERN Status
Tony Cass
The LCG Project is up and running
PEB, SC2 and POB have all met.
Launch Workshop, 11th-15th March
Hardware for 2002 Prototype is available
Scheduling ongoing.
RTAGs launched for Persistency, Managing Software, Maths Libraries and Grid Requirements.
Initial report from Persistency and Managing Software RTAGs at launch workshop.
RTAG reports will determine assignments for GridPP staff in relevant areas.
Fabric Management Activity Plan being prepared
Similarly, this will determine assignments for GridPP staff working on Fabric Management.
Tony Doyle - University of Glasgow

10. Tony Doyle - University of Glasgow
CERN concerns
Tony Cass
Ensuring compatibility between EU and US testbeds
Understanding impact if first physics moves to 2007
On Production system
On LCG Phase I
Scheduling of the Prototype
Understand needs and types of challenge - pushing the envelope or mass production for LHCC milestones
Fitting these in resources available
Tight schedule for new tape drives for ALICE
And we need to support running experiments in a reduced budget
Scheduling to allow test of Grid tools across multiple sites.
Tony Doyle - University of Glasgow

11. Tony Doyle - University of Glasgow
Technical Board
Pete Clarke
Composition to date (reflects dynamic nature):
Chair Pete Clarke
Deputy Robin Middleton
WP1 Dave Colling
WP2 Gavin McCance
WP3 Steve Fisher
WP4 Lex Holt/Paul Anderson
WP5 John Gordon
WP6 Andrew McNab
WP7 Richard Hughes-Jones
WP8 Frank Harris
Middleware Alex Martin
Experiments Steve Burke
Security Dave Kelsey
EB Nick Brook
Tony Doyle - University of Glasgow

12. Tony Doyle - University of Glasgow
TB Meetings
Pete Clarke
December
adoption of DataGRID reporting mechanisms for WP1-7 (3-monthly reports)
Jan 9th
EDG: Post quarterly submission review of delivered resources
EDG: Review of UK deliverable responsibilities
Gantt chart : testbed functionality
Testbed rollout planning: first discussion
Application builders workshop planning
adoption of
Feb 8th
Detailed discussion of Testbed rollout planning => SEE TOMORROW
First discussion of non-LHC exp support
GridPP DEVELOPMENT Grid
Tony Doyle - University of Glasgow

13. Tony Doyle - University of Glasgow
Resource Monitoring
Dave Britton
Next three-monthly
reports in March (~18th)..
Input = 3-monthly report
A work-segment is a structure something like:
{Quarter, Person, Institute, Experiment, Task, StaffMonths}
for example:
{2002Q1, P.Mealor, UCL, NONE, T7.2, 3.00}
Extract information such as:
1) Total delivered effort as a function of time.
2) Effort as a function of task or workgroup.
3) Effort as a function of Institute or Experiment (as a function of time).
4) Histograms of any of these quanitites (e.g. effort vs workgroup or Institute or experiment).
Technically - Input into Excel (visual basic macros)
Need to integrate into project management tools
Need to match to high level goals
Tony Doyle - University of Glasgow

14. Tony Doyle - University of Glasgow
TB Documentation
Pete Clarke
EDG resource management document produced and circulated
WP1-7 mechanisms now in place and being used usefully
Applications posts monitoring mechanisms now being reviewed
Testbed rollout planning and support document in preparation
iterated within TB
next step is circulation to EB and WPs affected (WP6) for comment
will then be circulated to all for comment
to be outlined by PC tomorrow
[Note: TB-WWW pages being fixed now]
Tony Doyle - University of Glasgow

15. Tony Doyle - University of Glasgow
TB Meetings
Pete Clarke
Application Builders meeting 31st RAL
aim was “how to use middleware” talks where appropriate (e.g. WP1)
in addition overview of what other middleware does
All future meetings will be as requested by GridPP (i.e. the audience)
Will probably only cover one theme
Next planned is half day (or more) on use of Job submission system
We will attempt to make the format more “training oriented” (perhaps use RAL or IC training rooms)
Will circulate request for interest (6 weeks from now)
Use other meetings, where appropriate e.g. RAL (SRB) and NeSC e-Science Institute (Globus)..
But try to minimise..
Tony Doyle - University of Glasgow

16. Tony Doyle - University of Glasgow

17. Project Management Board
Project Leader Tony Doyle Appointed by CB
Deputy Project Leader John Gordon Appointed by Project Leader
Chair of Collaboration Board Steve Lloyd Ex officio
Chair of Technical Board Peter Clarke Appointed by Project Leader
Chair of Experiments Board Nick Brook Ex officio
Deputy Chair of EB Roger Barlow Ex officio
Resource Manager Dave Britton Appointed by Project Leader
DataGrid UK Project Leader Robin Middleton Ex officio
CERN Liaison Tony Cass Appointed by CERN
e-Science Director Neil Geddes Ex officio
As proposal plus Resource Manager + DataGrid UK Project Leader + Part time Secretary at Glasgow
Three year posts (except EB 1+2)
Meets weekly via VC and monthly face to face
Tony Doyle - University of Glasgow

18. High Level Deliverables
To be re-defined
to extend to “The Grid”
i.e. testbed
Aspiration: all that you expect
from a central service?
Probably not..
Tomorrow’s Discussion..
Tony Doyle - University of Glasgow

19. High Level Deliverables
Requires input
from WP contacts
Tony Doyle - University of Glasgow

20. High Level Deliverables
Pre-defined
at high level,
but requires
further iteration
Tony Doyle - University of Glasgow

21. Projected Staff Effort [SY]
Area GridPP @CERN CS
WP1 Workload Management [IC] [IC]
WP2 Data Management [Ggo] [Oxf] WP12
WP3 Monitoring Services [RAL, QMW] 1.0 [HW]
Security [RAL] [Oxf]
WP4 Fabric Management [Edin., L’pool]
WP5 Mass Storage [RAL, L’pool]
WP6 Integration Testbed [RAL/M’cr/IC/Bristol]
WP7 Network Services [UCL/M’cr] 1.0 [UCL]
WP8 Applications
ATLAS/LHCb (Gaudi/Athena) [Oxf, Cam, RHUL, B’ham, RAL]
CMS [IC, Bristol, Brunel]
CDF/D0 (SAM) [IC, Ggo, Oxf, Lanc]
BaBar [IC, M’cr, Bristol]
UKQCD [Edin.]
Total >20.0? 6.0
Resource Management
Tony Doyle - University of Glasgow

22. DataGrid Recruitment Status
Robin Middleton
Dec 01
Tools being developed
Tony Doyle - University of Glasgow

23. Tony Doyle - University of Glasgow
DataGrid-UK Status
Robin Middleton
Recruitment
EU funded posts filled & in post (1 additional from 1/2/02)
All but 2 of 15 “unfunded” posts active
Reporting
UK templates defined, with TB meetings before & after quarterly submissions
81% of effort contracted delivered (but not uniform across WPs) for WP1-7 (WP8 hard to quantify)
UK Progress (in 2001)
Architectures, Testbed-1, network monitoring, certificates & security, R-GMA, LCFG, MDS deployment, Spitfire… …
Tony Doyle - University of Glasgow

24. Tony Doyle - University of Glasgow
DataGrid-UK Effort
Robin Middleton
Tony Doyle - University of Glasgow

25. Tony Doyle - University of Glasgow
DataGrid-UK Status
Robin Middleton
EU Review
All UK deliverables passed EDG internal review and now submitted to EU
Review at CERN on 1st March
3 external reviewers
~40 deliverables (inc. quarterly reports)
Summary presentations only
Can expect detailed feedback of areas to address
4th Project Conference – Paris 4th-8th March
Plans for 2002 – TB2
Absorb initial feedback from EU review
Tony Doyle - University of Glasgow

26. Tony Doyle - University of Glasgow
DataGrid-UK News
Robin Middleton
Licensing is still an issue
ATF reconstituted with more Applications involvement
New project web site at original address
Based at CERN
Uses EDMS for document handling
Testbed (see elsewhere at meeting)
Minor releases every 2 months
V2 in September
Transatlantic testbeds coming closer to reality
Scope being clarified
Interoperability a major issue  architectures
Tony Doyle - University of Glasgow

27. Tony Doyle - University of Glasgow
InterGrid etc
Robin Middleton
Strictly HENP-InterGrid
HICB meeting on 17th Feb Sunday (prior to GGF)
HIJTB meet monthly by phone
Projects
BaBar testbed
GRIT – Grid Reference International Testbed
ATLAS & CMS
DataTAG started
DataTAG : WP2=QoS, etc; WP4=m/w interoperability
Collaborates with iVDGL
CrossGrid, GridStart
Tony Doyle - University of Glasgow

28. Tony Doyle - University of Glasgow
GRIT
Ruth Pordes
Tony Doyle - University of Glasgow

29. “The Grid” is an evolving concept
enabler for transient ‘virtual organisations’
anatomy: a software infrastructure that enables flexible, secure, co-ordinated resource sharing among dynamic collections of individuals, institutions and resources
Foster, Kesselman & Tuecke (2001)
evolution of and integration with web services
physiology: everything is a Grid service i.e. a service that conforms to a set of conventions for management and exchanging messages
Foster, Kesselman, Nick & Tuecke (2002)
Global Grid Forum: define a standard Grid architecture
big business and big science starting to work together
GGF4 (Toronto) next week
Tony Doyle - University of Glasgow

30. Tony Doyle - University of Glasgow
GGF Architecture
1) scope the problem - what are we trying to produce
2) decide on the require functionality
3) modularize the functionality
4) decide on the relationships among the modules
5) identify the data that must flow among the modules
6) decide what state has to be maintained, and where
7) design the protocols for communicating among and with the modules
8) APIs
The challenge in the Grid forum is
- picked a large and amorphous problem (wide area distributed computing)
- for which there is not agreement (yet) as to what all of the functionality should be
- and even where we think that we understand the functionality requirements (e.g. data and resource management) there is not yet consensus on the modules
ANSWERS??
DataGrid++ is “The Architecture” we have adopted - look here first
Tony Doyle - University of Glasgow

31. Tony Doyle - University of Glasgow
e-Science and Grid
We are asked to map onto this plane…
Recognisable words, focussing on “middleware”
DataGrid++=middleware
(where ++=SAM, SLAC-based developments)
Experiments=Application Area
Middleware=Application Area??
Draft 2.0 status
Response should be built up from technical definitions
Tony Doyle - University of Glasgow

32. Tony Doyle - University of Glasgow
2002 Challenges
Complete rollout of TB-1 and plan future upgrades
Reconvened ATF to work closely with applications
Make TB-2 a success
Deploy and exploit Tier-1/A
Applications to make good use of testbeds
Solve interoperability issues
We are part of many larger collaborations/structures/groupings - we need to collaborate/discuss engage here, and
Focus on implementation in the UK… this will tell us what works (and what doesn’t) at any given point.
Tony Doyle - University of Glasgow

33. Status 13/2/02: Requires upgrades to functionality (extract hosts from
info. services
automatically)
and maintainability
(integrate into
testbed workplan)
Technical Management:
See tomorrow’s discussion
Tony Doyle - University of Glasgow

34. Tony Doyle - University of Glasgow
Status 13/2/02:
Expect that
Experiments will
deliver production
monitoring systems
e.g. AliEn for ALICE
(integrate into
testbed workplan)
Tony Doyle - University of Glasgow

35. Guidelines and Policies
Disk/CPU resources allocated to each “group”
Grid is based on distributed resources - a “group” is an experiment
An institute is typically involved in many experiments
Institutes define priorities on computing resources via OPEN policy statements
e.g. Edinburgh/Glasgow RAL ++
ScotGrid - Priority 1: ATLAS, LHCb, Computing Science Data Mining BaBar (2/3-1/2-1/3) LHC (1/3-1/2-2/3) tomorrow’s
- Priority 2: BaBar, CDF, UKQCD e-Science discussions
BioInformatics
All jobs submitted via Globus authentication - Certificates identified by user and experiment
Need to implement “priority”
Minimum amount of data to deliver at a time for a job?
Where to store files?
Which data access/storing activities have the highest priority?
Sharing of the resources among groups?
Users belong to multiple groups?
How many jobs per group are allowed?
What processing activities are allowed at each site?
To which sites should data access and processing activities be sent?
How should the resources of a local cluster of PCs be shared among groups?
REQUIRES Discussion at the Collaboration Meeting… issues will arise which require ALL Institutes to publish a policy
Longer term goals… Grid Credits issued?
Its all in the small print.
* Implemented by site administrators,
with OPEN policies defined at each site
based on e.g. case to funding authority
What’s new?
Ability to monitor/allocate unused resources
We will be judged by how well we work as a
Virtual Organisation
Tony Doyle - University of Glasgow

36. Tony Doyle - University of Glasgow
Summary
Most effort so far directed towards DataGrid=middleware
Significant UK contributions
Startup difficulties – largely overcome
Recruitment, late filling of posts – funded & unfunded
Lost ground being recovered..
All WP with UK involvement active & contributing
Testbed-1 : so far modest, predictable teething problems (many !)
Next major challenges focus on INTEGRATION
US experiments programme - planning for today/tomorrow..
LCG programme - planning for
Testbed (focus for tomorrow’s discussions) - requires input from system managers as well as software developers.
Positioned reasonably well ~6 months into GridPP..
with many challenges ahead...
Tony Doyle - University of Glasgow

37. Tony Doyle - University of Glasgow
Closing thought… 14/2/02
With many hidden words,
meanings, nuances…
“communication is the
answer to the problem”
(apparently)
Grids are everywhere..
Tony Doyle - University of Glasgow

38. 3rd Collab Meeting Agenda
Feb 14th
Opening Session (Chair - Andy Parker):
11:00-11:30 Welcome and Background - Steve Lloyd
11:30-12:00 GridPP Project Status - Tony Doyle
12:00-12:30 LHC Computing Grid Project Status - Tony Cass
Middleware Status Reports and Development (Chair - Tony Doyle):
13:30-13:50 Information Services (FTREE and MDS) - Alex Martin
13:50-14:10 Information Services (RGMA) - Manfred Oevers
14:10-14:30 Mass Storage - Jens Jensen
14:30-14:50 Fabric management - Michael George
14:50-15:10 Grid Data Management - Will Bell
15:10-15:30 Networking - Paul Mealor
Experiment Developments (Chair - John Gordon):
16:00-17:00 ATLAS, CMS, LHCb and theory - Nick Brook
17:00-17:30 WP8 status - Frank Harris
17:30-18:25 BaBar, CDF, D0, SAM and others - Roger Barlow
18:25-18:30 Logistics - Andy Parker
18:30-19:30 Project Management Board Meeting
Feb 15th
Cambridge Perspective and TestBed Introduction (Chair - Robin Middleton):
9:00-9:30 Cambridge e-Science Centre I - Andy Parker
9:30-10:00 Cambridge e-Science Centre II - Mark Hayes
10:00-10:30 TestBed Technical Development - Pete Clarke
Grid Deployment I (Chair - Pete Clarke):
11:00-11:30 Testbed Tools and Release in UK - Andrew McNab
11:30-11:40 Security issues - Linda Cornwall
11:40-11:55 Testbed status at RAL - Steve Traylen
11:55-12:10 Testbed status at M'cr - Andrew McNab
12:10-12:25 Testbed status at Bristol - Marc Kelly
12:25-12:40 Testbed status at IC - Dave Colling
Grid Deployment II (Chair - Steve Lloyd):
13:30-13:40 Testbed status at Edinburgh/Glasgow - David Martin
13:40-13:50 Testbed status at Liveropool - Girish Patel
13:50-14:00 Testbed status at Oxford - Pete Gronbech
14:00-14:10 Testbed status at UCL - Paul Mealor
14:10-14:20 Testbed status at Cambridge - Mark Hayes
14:20-14:30 Testbed status at Brunel - Peter van Santen
14:30-14:40 Testbed status at Lancaster - Roger Jones
14:40-15:00 Discussion
15:00-15:30 TestBed UK Organization - John Gordon
WELCOME!
Tony Doyle - University of Glasgow