Aug Arie Shoshani Particle Physics Data Grid Request Management working group

Aug Arie Shoshani parallel disk system Site Manager tape system parallel disk system Network Schematic of the distributed data system Applications Site Manager Site Manager tape system

Aug Arie Shoshani Request Manager Cache Manager Request Interpreter

Aug Arie Shoshani Scenarios Simple: All files for a local application are cached to a local disk-cache General: Files for an application can be cached to any disk-cache on any site

Aug Arie Shoshani Request Interpreter Storage Access service Request Manager Cache Manager Request to move files {file: from,to} logical request (property predicates / event set) Local Site Manager To Network File Access service Fig 1: Architecture for the general scenario - API boundaries files to be retrieved {file:events} Logical Index service Storage Reservation service Request to reserve space {cache_location: # bytes} Matchmaking Service File Replica Catalog GLOBUS Services Layer Remote Services Resource Planner Application (data request) Client (file request) Local Resource Manager Cache Manager Properties, Events, Files Index

Aug Arie Shoshani Functionality of modules (1) Request Interpreter accepts “estimate” requests based on “property predicates” accepts “execute” requests for: property predicates set of events set of files sends request inquiry to Logical Index Service consults Matchmaker + Resource Planner for estimation passes entire “logical file request” to Request Manager

Aug Arie Shoshani Functionality of modules (2) Request Manager accepts “logical file request” and notifies Application module “ready to process files” consults Matchmaker + Resource Planner to generate “request plan” accepts “get next events (files)” requests from Client module coordinates multi-application requests to share files issues “cache file(s)” request to Cache Manager notifies Client module when a (bundle of) files is available in some cache (can be on remote sites) notifies Cache Manager when a Client is finished with files

Aug Arie Shoshani Functionality of modules (3) Cache Manager accepts “cache a file” requests from the local Request Manager accepts “reserve space” requests from remote reservation calls allocates space on local cache as well as tape resources according to a local resource management policy issues “lock a file” service requests to remote sites issues “reserve space” service requests to remote sites issues “move a file” service request issues “release a file” service request to remote sites notifies Matchmaker of any local cache changes

Aug Arie Shoshani Functionality of modules (4) Local Resource Manager accepts requests for multi-file caching Determines the best way to allocate local tape and disk cache resources notifies cache manager to reserve space notifies tape resource manager to reserve tape drives...

Aug Arie Shoshani Request Interpreter Storage Access service Request Manager Cache Manager Request to move files {file: from,to} logical request (property predicates / event set) Local Site Manager To Network File Access service Fig 1: Architecture for the simple scenario files to be retrieved {file:events} Logical Index service Storage Reservation service Request to reserve space {cache_location: # bytes} Matchmaking Service File Replica Catalog GLOBUS Services Layer Remote Services (Resource Planner) Application (data request) Client (file request) Local Resource Manager Cache Manager Properties, Events, Files Index To Local Cache

Aug Arie Shoshani Request Interpreter Storage Access service Request Manager Cache Manager Request to move files {file: from,to} logical request (property predicates / event set) Local Site Manager To Network File Access service Fig 1: Architecture for the general scenario - APIs use order files to be retrieved {file:events} Logical Index service Storage Reservation service Request to reserve space {cache_location: # bytes} Matchmaking Service File Replica Catalog GLOBUS Services Layer Remote Services Resource Planner Application (data request) Client (file request) Local Resource Manager Cache Manager Properties, Events, Files Index

Aug Arie Shoshani Sequence of API calls 1. Application issues an estimate 2. Consult Logical index Service (properties-events-files) for estimate 3. Consult Matchmaker + Resource Planner for estimate 4. Issue “execute” request command 5. Consult Logical index Service for precise request evaluation 6. Sent {files:events} request to Request Planner 7. Client issues a request for next events to process (files are implied) 8. Request Planner Consults Matchmaker + Resource Planner 9. Request Planner issues “get file(s), source, destination” request 10. Cache Manager issues “reserve space” request if “destination” is remote site 11. Cache Manager issues “move file(s)” request 12. Cache Manager notifies Matchmaker of any changes to local cache 13. Client issues commands to read events out of files

Aug Arie Shoshani People assigned to define API interfaces (first name responsible to contact others) Alex, Dave, Vicky (Igor) Vicky (Igor) to coordinate with Miron’s working group 6 -- Alex, Arie 7 -- Dave, Vicky (Igor), Andy 9 -- Luis, Andy (for bundle at a time request, not just a single file) Luis, Andy Andy, Luis Andy, Dave

Aug Arie Shoshani Request Interpreter Storage Access service Request Manager Cache Manager Request to move files {file: from,to} logical request (property predicates / event set) Local Site Manager To Network File Access service files to be retrieved {file:events} Logical Index service Storage Reservation service Request to reserve space {cache_location: # bytes} Matchmaking Service File Replica Catalog GLOBUS Services Layer Remote Services (Resource Planner) Application (data request) Client (file request) Local Resource Manager Cache Manager Properties, Events, Files Index To Local Cache Fig 3: OOFS integration OOFS Objectivity AMS Event (File) in Cache? Reserve / make space Move file HPSS-DISK Where OOFS functionality corresponds to architecture components