OGSA Data Architecture Scenarios Dave Berry & Stephen Davey GGF18, 13th September 2006

Contents Overview Five sample scenarios 2

Two Informational Documents OGSA Data Architecture 70+ pages Describes the services and their interfaces Some work remaining to describe interfaces OGSA Data Scenarios 50+ pages Describes how the services can be combined to address particular scenarios Some work remaining to identify interfaces 3

Scenarios document Example scenarios of a generic nature to accompany the OGSA Data Architecture document. Illustrates how the components and interfaces described in the OGSA Data Architecture document can be put together in a selection of typical data scenarios. Not a use case document generating requirements. 4

Current Scope Files and databases (& storage) Not streams, sessions, … Services and interfaces Storage, Access, Transfer Replication, Caching, Federation, Metadata catalogues Cross-cutting themes Security, Policies, … Part of the bigger OGSA picture E.g. Naming, Workflow, Transactions, Scheduling, Provisioning, … 5

Progress since GGF16 More scenarios More integration E,g, Provenance, Grid File System More integration Particularly between scenarios and architecture document Also raising some issues from individual chapters to cross-cutting concerns 6

Data pipelining Completed Animations Visualisation Service Customer 2 1. Submit job. 2. Store results. 3. Transfer results. 4. Return results. 1 Data Transfer Rendering Data Access 7

Bringing data online Data Storage Service Customer 1. Make files online. 1. Make online. Nearline Storage 1. Make online. 3. Retire to nearline. 3. Retire to nearline. 2. Read files. The client has the file names for a set of files in a given space and requires that these files should be available online. The files are made available online by the Data Storage Service. The data are read through an appropriate interface, such as the Transfer Service. The online attribute of the files may expire and they can be retired to nearline storage. The interface to the Data Storage Service needs to be able to provide the ability to specify that the data should be available with low-latency and high bandwidth for a certain amount of time. This ‘online time’ concept is independent of the total data lifetime in the storage and needs to be managed separately. However, it cannot exceed the total data lifetime. Transfer Service Online Storage 2. Read files. Storage Devices 8

Replication Data Service 1 Data Storage 1 4. Access data Customer 1a. Register data 2. Transfer copies 5. Notify Data Transfer Service Customer 2 Replication Service 6. Update 2. Transfer copies A data resource is registered with a replicating data service (details such as creation time, access control, etc. would also be included) and replication service enters the data resource into a replica catalogue. The replication service uses a data transfer service to move copies of this data to different locations and tracks which data is kept where. Clients access the catalogue to find the data resource, or to return a list of resources that satisfy certain Quality of Service (QoS) requirements. Clients then access the stores either directly or indirectly. Changes to the data are notified to the replication service. Updates then occur between the data services to synchronize the replicas. 2. Transfer copies 3. Find data 1b. Publish Data Storage 2 Data Service 2 Registry Service 9

Data Staging Input Data Output Data (copy) Data Transfer Service 2a. Data Service 1 2b. Transfer input data. 4a. Stage output data. 2a. Stage input data. 4b. Transfer output data. Client 2a. 4a. 1. Submit JSDL script. Data Service 2 BES Container BES Container: 3. Run executable & save resulting output data. 5. Delete input data. 6. Delete output data. Output Data Input Data (copy) 10

Global Name Resolver Service Personal Data Service Registry Service Data Service 1 1. Locate data. Customer 1 (site 1) 2. Create named space. Local Cache Service 1 2. Create. 4. Use named space. Data Service 2 5. Update. Personal Data Service Index Data Service 3 7. Update. Customer at site 1 locates data by using, for example, a Registry Service. The Customer interacts with the Personal Data Service (via their Local Cache Service) in order to create a personal collection of data (a named space). The Personal Data Service uses a Global Name Resolver Service in order to name the customer’s collection of data. The Customer at site 1 uses Local Cache Service 1 in order to build and modify their personal collection of data. On terminating the session at site 1 the Local Cache Service 1 updates the Personal Data Service. Customer moves to site 2 and starts working, wanting to use, change and add to their personal collection of data. This is done via Local Cache Service 2. On terminating the session at site 2 the Local Cache Service 2 updates the Personal Data Service. Index Customer 1 (site 2) 6. Use named space. Local Cache Service 2 3. Name collection. Global Name Resolver Service Index 11

Questions?