1. OGSA Data Architecture Scenarios
Dave Berry & Stephen Davey
GGF18, 13th September 2006

2. Contents
Overview
Five sample scenarios 2

3. Two Informational Documents
OGSA Data Architecture
70+ pages
Describes the services and their interfaces
Some work remaining to describe interfaces
OGSA Data Scenarios
50+ pages
Describes how the services can be combined to address particular scenarios
Some work remaining to identify interfaces 3

4. Scenarios document
Example scenarios of a generic nature to accompany the OGSA Data Architecture document.
Illustrates how the components and interfaces described in the OGSA Data Architecture document can be put together in a selection of typical data scenarios.
Not a use case document generating requirements. 4

5. Current Scope Files and databases (& storage) Not streams, sessions, …
Services and interfaces
Storage, Access, Transfer
Replication, Caching, Federation, Metadata catalogues
Cross-cutting themes
Security, Policies, …
Part of the bigger OGSA picture
E.g. Naming, Workflow, Transactions, Scheduling, Provisioning, … 5

6. Progress since GGF16 More scenarios More integration
E,g, Provenance, Grid File System
More integration
Particularly between scenarios and architecture document
Also raising some issues from individual chapters to cross-cutting concerns 6

7. Data pipelining Completed Animations Visualisation Service Customer 2
1. Submit job.
2. Store results.
3. Transfer results.
4. Return results. 1
Data Transfer
Rendering
Data Access 7

8. Bringing data online Data Storage Service Customer
1. Make files online.
1. Make online.
Nearline
Storage
1. Make online.
3. Retire to nearline.
3. Retire to nearline.
2. Read files.
The client has the file names for a set of files in a given space and requires that these files should be available online.
The files are made available online by the Data Storage Service.
The data are read through an appropriate interface, such as the Transfer Service.
The online attribute of the files may expire and they can be retired to nearline storage.
The interface to the Data Storage Service needs to be able to provide the ability to specify that the data should be available with low-latency and high bandwidth for a certain amount of time. This ‘online time’ concept is independent of the total data lifetime in the storage and needs to be managed separately. However, it cannot exceed the total data lifetime.
Transfer Service
Online
Storage
2. Read files.
Storage Devices 8

9. Replication Data Service 1 Data Storage 1 4. Access data Customer
1a. Register data
2. Transfer
copies
5. Notify
Data Transfer
Service
Customer 2
Replication
Service
6. Update
2. Transfer
copies
A data resource is registered with a replicating data service (details such as creation time, access control, etc. would also be included) and replication service enters the data resource into a replica catalogue.
The replication service uses a data transfer service to move copies of this data to different locations and tracks which data is kept where.
Clients access the catalogue to find the data resource, or to return a list of resources that satisfy certain Quality of Service (QoS) requirements.
Clients then access the stores either directly or indirectly.
Changes to the data are notified to the replication service.
Updates then occur between the data services to synchronize the replicas.
2. Transfer
copies
3. Find data
1b. Publish
Data Storage 2
Data Service 2
Registry Service 9

10. Data Staging Input Data Output Data (copy) Data Transfer Service 2a.
Data Service 1
2b. Transfer input data.
4a. Stage output data.
2a. Stage input data.
4b. Transfer output data.
Client
2a.
4a.
1. Submit JSDL script.
Data Service 2
BES Container
BES Container:
3. Run executable & save resulting output data.
5. Delete input data.
6. Delete output data.
Output Data
Input Data
(copy) 10

11. Global Name Resolver Service
Personal Data Service
Registry
Service
Data
Service 1
1. Locate data.
Customer 1 (site 1)
2. Create named space.
Local Cache Service 1
2. Create.
4. Use named space.
Data
Service 2
5. Update.
Personal Data Service
Index
Data
Service 3
7. Update.
Customer at site 1 locates data by using, for example, a Registry Service.
The Customer interacts with the Personal Data Service (via their Local Cache Service) in order to create a personal collection of data (a named space).
The Personal Data Service uses a Global Name Resolver Service in order to name the customer’s collection of data.
The Customer at site 1 uses Local Cache Service 1 in order to build and modify their personal collection of data.
On terminating the session at site 1 the Local Cache Service 1 updates the Personal Data Service.
Customer moves to site 2 and starts working, wanting to use, change and add to their personal collection of data. This is done via Local Cache Service 2.
On terminating the session at site 2 the Local Cache Service 2 updates the Personal Data Service.
Index
Customer 1 (site 2)
6. Use named space.
Local Cache Service 2
3. Name collection.
Global Name Resolver Service
Index 11

12. Questions?