1. IVOA Interop, Beijing, China, May 20071 IVOA Data Access Layer Working Group Sessions Doug Tody (NRAO/NVO ) Markus Dolensky (ESO/EuroVO) Data Access Layer Working Group I NTERNATIONAL V IRTUAL O BSERVATORY A LLIANCE

2. IVOA Interop, Beijing, China, May 20072 Advanced Service Operations stageData

3. IVOA Interop, Beijing, China, May 20073 DAL Service Profile Standard Profile –Intended as starting point for any 2ndGen DAL service –Certain operations require GWS technology, or were motivated by efforts such as VOSI Standard Operations –queryData find available datasets (may be virtual) –(getData) get a single dataset (synchronous) –stageData start async job to generate/stage datasets –getCapabilities get service metadata (capabilities) –getAvailability get service availability, status

4. IVOA Interop, Beijing, China, May 20074 DAL Service Profile QueryData –Describes multiple datasets which can be retrieved In the future, a more complex query (e.g., multi-region) may describe many datasets –Data may of course be virtual data, requiring potentially significant computation to produce Hence a "dataset" represents a computational task A synchronous GET can be used to access data Large computations or batch mode requires stageData –Can include cost estimate information –Can iterate query to refine computational parameters

5. IVOA Interop, Beijing, China, May 20075 StageData Operation Concept –StageData is the "job submission" mechanism –Primary input is a list of datasets to be generated can be referred to by PubDID or acref job can be any size (thousands of concurrent tasks possible) –Includes information on data disposition (e.g., for VOSpace) How Might it Work? –Implemented as a POST –Response is status info and a JobID –All further communication based on service baseURL and JobID –Can use standard UWS job mechanism once job initiated –Could also be used to send commands to job once it is running

6. IVOA Interop, Beijing, China, May 20076 StageData Operation Monitoring Job Status –Two approaches Polling (a GET to some status URL based on baseURL, JobID) Event stream (a streaming GET used to carry event messages) –Summary Both are worth having Polling is simplest, should be mandatory Asynchronous messaging not difficult, much more powerful

7. IVOA Interop, Beijing, China, May 20077 Monitoring Job Status Polling –Simple, can support multiple clients –High latency, does not scale well, not terribly efficient –Can only see current status (no events, state transitions) Event Stream –Implemented as a streaming GET Current status transmitted at connect time, updates thereafter Status packet (e.g., %done) transmitted periodically –Still simple; can support multiple clients –Can disconnect and later reconnect –Any event can be reported to the client immediately –Low latency, efficient –Can add interactivity to any service

8. IVOA Interop, Beijing, China, May 20078 Monitoring Job Status Sample Events –Status (%done/keep-alive; cadenced) –Information (verbose/debug messages) Typical Job Events –jobStarted, jobCompleted, jobError –taskCompleted (data object finished and ready) –dataDelivered (can include acref)