David Adams ATLAS AJDL: Analysis Job Description Language David Adams BNL December 15, 2003 PPDG Collaboration Meeting LBL

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Contents Analysis AJDL Data object types Service functions Conclusions

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Analysis service Functionality User specifies application, task and dataset Service processes and returns result User examines result, modifies task or changes dataset and resubmits User monitors job during processing User has option to kill job Common interface Provides the above Allows generic service to be shared –Robust, efficient analysis service is difficult

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, User Analysis Job 1 Job 2 ApplicationTask Dataset 1 Analysis Svc 1. Create or locate 2. select3. Create or select 4. select 5. submit(app,tsk,ds,cfg) 6. split Dataset Dataset 2 7. create e.g. ROOT e.g. athena Result 9. fill 10. gather Result 9. fill ResultCode Analysis service

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, AJDL as a bridge

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, AJDL Analysis Job Description Language (AJDL) includes Data object types –E.g. could be expressed in XML Analysis service functions taking objects as arguments –E.g. SOAP messages For now, we are more concerned with the content of these objects and messages Rather than the format However, need format for implementation DIAL WSDL and DTD is close

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Data object types Application Task Dataset (and subtypes) Configuration Result (and subtypes?) Job

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Application Purpose Specify the application executable Content Name and version (of application SW package) Analysis service usage Locate script to build task Locate script to process dataset and generate result Package management service Provides location of app package (name, version) May install app package and dependencies –Exe is likely in separate package

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Task Purpose Allow user to configure application Content Collection of named files –Embedded text or –Logical files Files contain parameters and code Analysis service usage Use with script to build task Stage files for building task

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Dataset Purpose Specify and split data for processing Content Description used by application processing script –Dataset subtype Event count and ID list –EventDataset List of constituent datasets –CompoundDataset Abstract dataset specification –VirtualDataset (no location) –QueryDataset

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Dataset (cont) Analysis service usage Used with script to process data and generate result –Pass dataset to script Split dataset to create sub-jobs –EventDataset to split along event boundaries –CompoundDataset provides guidance for splitting Find concrete dataset for abstract specification –Use dataset selection service for QueryDataset –Use dataset replica service for VirtualDataset

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Configuration Purpose Specify how to process dataset Content User identity and role Response time –Desired –Maximum acceptable Accounting –Account to charge –Budget

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Result Purpose Communicate results to user Content Datasets Named files –Embedded or logical –For arbitrary data >E.g. collection of histograms –Or use Result subtype?

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Job Purpose Communicate job status and results to user Content Application, task, dataset and configuration Result –Possibly partial Status –Running, done, failed, … Start, stop and update times

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Service functions Analysis service includes bool has_application(Application) bool has_task(Application, Task) bool install_task(Application, Task) JobId submit(Application, Task, Dataset, Config) Job job(JobId) bool kill(JobId)

David Adams ATLAS AJDL PPDG All Hands – uJDL sessionDecember 15, Conclusions Standard analysis service interface Allow common service to be shared First pass presented here More information AJDL document DIAL JDL page