1. GridPP9 – 5 February 2004 – Data Management DataGrid is a project funded by the European Union GridPP is funded by PPARC GridPP2: Data and Storage Management Gavin McCance - University of Glasgow Jens Jensen - RAL GridPP9, NeSC, Edinburgh

2. GridPP9 – 5 February 2004 – Data Management – n° 2Gavin McCance – University of Glasgow GridPP2 Middleware Data and Storage Management

3. GridPP9 – 5 February 2004 – Data Management – n° 3Gavin McCance – University of Glasgow Work areas u UK metadata management group u Storage management

4. GridPP9 – 5 February 2004 – Data Management – n° 4Gavin McCance – University of Glasgow Metadata Management u The focus is upon Grid-enabling metadata services for the experiments n Building upon our previous work in this area n Building upon experiments existing work in this area u Formation of a UK metadata group with GridPP2 n 1 generic Grid metadata post @ Glasgow n ~1 post per experiment s ATLAS @ Glasgow, LHCb @ Oxford, CMS @ Bristol/IC US expts, others?? s These posts were described yesterday – the UK metadata group should form part of their work n Input from the UK data management support teams

5. GridPP9 – 5 February 2004 – Data Management – n° 5Gavin McCance – University of Glasgow GridPP2 Metadata Group u Purpose will be to n Take overall responsibility for common experiment metadata technologies in order to Grid-enable the experiments metadata n Identify the commonalities and experience across experiments and make sure these are recognized s i.e. technologies, schema: data product navigational problem n Come to agreement and feed this back into the wider ARDA process u Work directly with interested groups forming the ARDA n EGEE JRA1 Data Management Group (@CERN) n LCG Deployment Teams (@CERN) n LCG Experiments n IT Database group (@CERN)

6. GridPP9 – 5 February 2004 – Data Management – n° 6Gavin McCance – University of Glasgow Metadata Responsibilities u Generic metadata post: n Concentration on the technologies used to create scalable, manageable and fault-tolerant metadata services s The underlying Grid software stack n Emphasis upon the service, not just the product s 24/7 supportable production services n Not prescribing things like the schema, or saying the API must look like Spitfire: prototype interfaces should be based upon experiments existing metadata interfaces n Will track, develop and adopt as necessary Grid metadata access standards s Feed into standards to make sure were in a position to benefit from the future production products that implement these standards s Feed PPE use-case and experience back into the wider world

7. GridPP9 – 5 February 2004 – Data Management – n° 7Gavin McCance – University of Glasgow Metadata Responsibilities u Experiment metadata posts (~1 per experiment): n Document existing implementations from the experiments and make sure all the experiments use-cases are satisfied by the products and the technologies being proposed by the group n Work within the group to ensure that commonalities and experience across experiments are recognized and effort is not wasted s At the technology level – e.g. using the same underlying Grid software stack s At the interface level – e.g. GANGA s Possibly at the schema level… n Feed this understanding and agreement back into the wider ARDA process and back into their own experiments n ARDA terminology: Dataset metadata ARDA Metadata service Data product navigation ARDA Job Provenance service

8. GridPP9 – 5 February 2004 – Data Management – n° 8Gavin McCance – University of Glasgow Storage Management u Two areas of work (based at RAL) u SRM interface to UK storage sites u Site local data management

9. GridPP9 – 5 February 2004 – Data Management – n° 9Gavin McCance – University of Glasgow SRM interface to UK Storage u Initial deliverable will be to provide an SRM (Storage Resource Manager) v1 interface to the Atlas DataStore at RAL n Subsequent migration to the more advanced features offered by e.g. SRM v2 u Perform an analysis of the UK Tier-2 storage sites and how these can be exposed via the common SRM interface n Implementation of SRM interfaces these storage systems n Deployment on all the Tier-2 sites and support u Contribution to the SRM standardisation process u Work closely with the EGEE JRA1 and LCG deployment groups u Work with support staff for Tier-1 and Tier-2

10. GridPP9 – 5 February 2004 – Data Management – n° 10Gavin McCance – University of Glasgow Site-local Data Management u Management of data and files within a site n How you access the grid storage from the worker nodes n Cleanup of volatile data resources that a job no longer needs (Tier2) – cache management u Evaluation of existing technologies n dCache, SAM, EDG Zambo prototype, Condor, … u Development and deployment of these local data management solutions (@ Tier-2) n Interaction with Tier-2 site managers is vital u Feed back solutions into LCG / EGEE

11. GridPP9 – 5 February 2004 – Data Management – n° 11Gavin McCance – University of Glasgow GridPP2 Support Data and Storage Management

12. GridPP9 – 5 February 2004 – Data Management – n° 12Gavin McCance – University of Glasgow Data Management Support u UK data management support posts n Aim: to provide first-level support for all DM software s first stop for UK system administrators n Work directly with the development and deployment teams (GridPP2, EGEE and LCG) n Provide hands-on deployment help for data challenge support n Develop how-to portal to collect deployment experience n Feed back sys-admin issues and experience to developers s Site policies, quotas, firewalls – survey sysadmins n Develop site validation tools n Responsible for developing the overall support plan for the data management services beyond GridPP2 n Need to fit all this in with the rest of the UK Support Plan