1. agINFRA science gateway for workflows and integrated services 07/02/2012 Robert Lovas robert.lovas@sztaki.mta.hu MTA SZTAKI

2. Why workflows? ‘Orchestration of Tasks’ (not only in sequence; organized in Direct Acyclic Graphs) Data-driven To exploit large computational resources / process large data sets To make the complex applications run faster –By applying paralellization techniques on them Paralellization Techniques: –Indepent tasks can be executed concurrently –Execute tasks against LARGE datasets  parameter study, domain decomposition, etc.

3. Parameter study / domain decomposition GEN SEQ COLL SEQ Generates input parameter space Evaluates the results of the simulation Parameter sweep jobs 3

4. Liferay-based WS-GRADE/gUSE portal for agINFRA: –http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/http://aginfra-portal.lpds.sztaki.hu/liferay-portal-6.0.5/ –Open Registration for project participants –X509 Certificate authentication required to be able to submit jobs  NEW: robot certs –agINFRA VO is accessible (ca. 5000 CPU cores, >50 TB storage)

5. Access modes 5 ASM API WS-PGRADE UI Customized UI Other, existing UI gUSE Workflow engine agINFRA VO Volunteers’ computers

6. Workflow building blocks (glossary) “Jobs” operating on data Jobs can be: –Grid-enabled applications (e.g. AgrovocTagging) –Web-services –NEW: REST services –Another workflows (embedded) “Ports” representing inputs and outputs for the jobs –Available port types: Value Local file e.g. from the scientists laptop Remote File (gsiftp, lfc) in the Grid Database Queries (SQL) …. –Extensions or improvement might be required (e.g. Drupal / Dublin Core / SPARQL / CIARD RING support?) 6

7. agINFRA overview

8. Services to be integrated Harvesting, validation, transformation

9. The Organic.Edunet Ingest Workflow

10. Schematic Representation of the AGRIS workflow

11. Cross-community workflows identified at the Athens

12. DEMONSTRATION I.

13. AgroTagger

14. First demo application

15. First demo application - details

16. Job details

17. Inputs and outputs

18. Monitoring of execution 18

19. Successful execution of AgrovocTagging application

20. DEMONSTRATION II. Harvesting workflow

21. ARIADNE aggregation panel - Select scheduling Link to the workflow interface -Invoke the workflow -Check the status of the workflow -Stop the workflow -Add metadata for the aggregation ARIADNE aggregation panel - Select scheduling Link to the workflow interface -Invoke the workflow -Check the status of the workflow -Stop the workflow -Add metadata for the aggregation gUSE WS-PGRADE Harvesting - Add parameters of agDataHarvesters web service Harvesting - Add parameters of agDataHarvesters web service Metadata Validation vs target schema - Add parameters of agMetadataValidation web service Metadata Validation vs target schema - Add parameters of agMetadataValidation web service Target Validation - Add parameters of agTargetValidation web service Target Validation - Add parameters of agTargetValidation web service Target schema? No Stop the process for the specific target send message or store logs and send them through the gUSE API Yes Transformation - Get parameters for the transformation and invoke agMetadataTransformation web service Transformation - Get parameters for the transformation and invoke agMetadataTransformation web service No Store metadata on the GRID (agINFRA VO) Store metadata on the GRID (agINFRA VO) Yes Valid? starting a pre-defined procedure as an agINFRA workflow

22. Integration of multiple components/services

23. Details in demo…

24. Plan: Continue the development of aggregation workflow

25. Plan: Further integration + volunteers…

26. 07/02/2012 Questions? 26