1. EGEE is a project funded by the European Union under contract IST-2003-508833 Data Management Gaps Krzysztof Nienartowicz Gavin McCance EGEE JRA1 Data Management Cluster JRA1 Meeting, Oxford, 30 June-2004 www.eu-egee.org

2. JRA1 Meeting, Oxford, 30-June-2004 - 2 Functional Gaps File pinning on SE  Who asks the SE to pin/unpin the files? For how long?  The DS or the site job manager? Optimization  Is any data optimization necessary? Using the replica catalog to inform the WMS for optimal job placement? Use replica catalog as store for current state of transfer? Use replica catalog as store for staging status? Implies continuous link between SE and catalogs.

3. JRA1 Meeting, Oxford, 30-June-2004 - 3 Gaps: Volume, scalability How many items: 10 6- 10 9 ?  Logical space in (file) catalogs  Over SEs Partitioning of data – both logical and physical dimensions  Over sites/nodes/groups/users/sites? Job limitations  Functional difference depending on the transactional logic: Is partial success of SE work acceptable by Job Manager? JDL option? Max size of transactions or bulk operations Max transfer times Max error occurrences of specific type?  Contingency logic if we hit limits Policy management, quota policy as a JDL option/hint?

4. JRA1 Meeting, Oxford, 30-June-2004 - 4 Gaps Distribution of catalogs  Necessary?  Propagation of changes Local transactions Model for replication, updates –Lazy updates –Publish-subscribe –Relationship with Accounting/Auditing Clashes handling –Manual –Semi-manual –Semi-automatic –Automatic

5. JRA1 Meeting, Oxford, 30-June-2004 - 5 Gaps- Complex Interface implementation. Internal dependencies, atomicity.

6. JRA1 Meeting, Oxford, 30-June-2004 - 6 Gaps- Complex Interface implementation. Cost of modularisation.