1. Univerza v Ljubljani Fakulteta za gradbeništvo in geodezijo Open Workflow Infrastructure: A Research Agenda Vlado Stankovski, Paolo Missier, Carole Goble, Ian Taylor WANDS workshop @SIGMOD 2010 Indianapolis, June 6, 2010

2. 2 Introduction University of Ljubljana, Faculty of Civil and Geodetic Engineering, Department of Construction Informatics Assist. Prof. in Computer Science Research interests Semantic grid Workflow technologies

3. 3 Complex engineering problems Highly structured and/or massive data product models, accellerograms,.. Heterogeneous devices available on the Internet processors, clusters, storage, sensors IPv6 Computationally intensive algorithms highly specialized, new developments Finite Elements, simulations, data mining Collaboration needs knowledge exchange within the engineering community dynamic integration of distributed resources

4. 4 Some end-user requirements A system for distributed applications should Support development by domain specialists, who are not IT experts Hide details of the underlying infrastructures from the end-users Open resource search and discovery is possible Integrate users, services, resources, workflows beyond a single organization boundary Allow secure access to internal infrastructure Support scenarios to offer “software, data, workflows,.. as a service” ID management, different user types is possible Scalability, performance needed; data or process intensive

5. 5 Workflows and SOA Workflow Editor & Manager Workflow Editing Workflow Manager V2V2 V3V3 VNVN V4V4 2 Information Service Ontology Service V6V6 V1V1 V5V5

6. 6 Research topics Independence from the underlying Distributed Computation Platforms (clusters, servers, grid/cloud infrastructures, peer-to-peer networks) Customization Harvesting and harnessing the intelligence of the communities Security support, ID management and provenance

7. 7 Workflows Raise the level of abstraction when developing distributed applications Accessible to non-programmers knowledge processescommunicationstoragecontent More effective means for sharing the knowledge, processes, communication, storage and content, then individual services

8. 8 Workflow systems A douzen workflow systems have been developed in the past years Triana, Taverna, Askalon, Kepler, Windows WF Foundation, Discovery Net etc. They address complex problems in a range of scientific and engineering fields However, a comprehensive methodology or tool that facilitates open, generic and rapid development of distributed applications represented through the workflow methodology, is still missing

9. 9 Taverna

10. 10 Askalon

11. 11 Kepler

12. 12 Windows WF Foundation

13. 13 Recent trends for open workflow development Yahoo Pipes http://pipes.yahoo.com IBM QEDWiki http://services.alphaworks.ibm.com/qedwiki Google Mashup Editor http://code.google.com/gme/ Microsoft Popfly www.popfly.com

14. 14 Technology convergence eScience workflows Business Process Modelling

15. 15 Research agenda Towards a science gateway Unified views on workflow system capabilities Web-wide search and discovery of resources (workflows, data and services) Extending collaborative practices in science and industry

16. 16 Distributed Audio Retrieval Community Medical & bioinformatics communities AEC Professionals Community Infrastructures Enabler (for infrastructure operators) Application Usage Knowledge Generation Social Networking Knowledge Generation Taverna WF EnactorTriana WF Manager Domain Knowledge Infrastructures & environment Services Registry Application Usage, Provenance Data Registry Workflow hosting and service interfaces Interfaces to other WF systems Collaboration Spaces (A Science Gateway) Data services Security, ID management InfrastructuresServices (security, logging, monitoring etc.) Resources, middleware services, clusters, and other DCPs Knowledge Layer Metadata schemata Software utilities Workflow Composition Modules (for workflow developers) Data Explorer, Service Explorer, Workflow Explorer and so on. (for end-users) Operational Research Proofs of concept from projects like: OntoGrid InteliGrid DataMiningGrid K-WfGrid Discovery Net..

17. 17 End-users Engineers, scientists Enhanced interfaces for resource search and discovery Workflow developers Modules for workflow development, sharing and use Modules for the management of community shared services and infrastructures System administrators Distributed metadata and knowledge sharing system Data management services Overlay security and identity management services Provenance Services

18. 18 Workflow composition modules Application Explorer (A) Data Manipulators (B) Parameter Control (C) Execution and Monitoring Units (D) Provenance Unit (E) E

19. 19 An Internet-wide Application and Service Explorer Application available at University of Ljubljana, Slovenia Applications available at Institute Josef Stefan, Slovenia Applications available at Fraunhofer Institute, Germany

20. 20 Manipulating distributred data resources files, databases, directories Data available in Skopje, Macedonia Data available in Belfast, UK

21. 21 Parameter Control Dynamically formed GUI metadata describing the end-user interface Used to set up execution parameters

22. 22 Available computing clusters are found in Slovenia, Ireland and Germany Resource brokering and execution monitoring

23. 23 Provenance Collects data about workflow execution where? when? which algorithm? what parameters? what data? who? for which application domain? Positive and negative examples

24. 24 Lokacija Število procesorjev Univerza v Ulstru 50–70 Univerza v Ulstru 4–10 Nemčija, inštitut Fraunhofer 4 Univerza v Ljubljani 20–40

25. 25 Research agenda Large scale integration of resources, services and infrastructures Identify infrastructures and define models for their integration, sharing and use Define public interfaces for workflow enactment Metadata schemata Design and implement annotation pipeline for harnessing collective (domain) intelligence Describe applications and services in uniform way Describe data Provenance models for distributed applications

26. 26 Key question How to make the knowledge about distributed applications explicit so that it can be shared within and among the communities? Workflow technologies have the potential to address many of the identified requirements

27. 27 Thank you! Q&A

28. 28 Delotoki Ian Taylor in sod., 2007 “Delotok je zaporedje povezanih storitev, ki se izvaja v primerno orkestriranih korakih” Povezujejo podatke, analitična orodja in storitve za simulacijo in vizualizacijo rezultatov/modelov

29. 29 Konvergenca tehnologij.. z dveh področij: Modeliranje poslovnih procesov Znanstveni in inženirski izračuni

30. 30 WS-BPEL 2.0 Bogat jezik za orkestracijo tako poslovnih kot tudi znanstvenih in inženirskih delotokov BPEL proces določa natančni vrstni red klicanja sodelujočih storitev Omogoča izdelavo zank, deklaracije spremenljivk, določanje vrednosti, obravnavanje izjem/napak, paralelno in zaporedno izvajanje itd. Ne omogoča pogojno izvajanje zank! Podpira razvoj ACID (Atomicity, Consistency, Isolation, Durability) transakcij Večina sistemov, ki podpira BPEL omogoča tudi persistentna stanja delotokov

31. 31 Project proposals YouCreate: Create, Share and Enact Pervasive Workflow-Centric Applications with Colleagues and Friends WF-Tube: A Workflow Tube WorkflowOmnibus, OurFactory... and so on.

32. 32 Odprta arhitektura

33. 33 Triana Urejevalnik in upravljalec delotokov grafično podprto prožno enostavno sestavljanje in izvajanje delotokov odprtokodna rešitev http://trianacode.org http://trianacode.org

34. 34 Triana podpira (povzetek) Spletne (SOAP, WSDL) in mrežne (WSRF) storitve Storitve tipa enak z enakim (angl. Peer-to- Peer) (Jxta, P2PS) mrežne storitve preko sistema Grid Application Toolkit (npr. SAGA) storitve za odkrivanje znanja v podatkih (npr. DataMiningGrid) lokalna Java orodja in izvršljive programe prožen dostop do HTTP storitev prepletene oz. hibridne (angl. mash-ups) storitve so hkrati enostavne in trivialne za implementacijo

35. 35 Upravljalec delotokov

36. 36 Light clients

37. 37 Zaključek kompleksne aplikacije tehnologije delotokov omogočajo večjo kontrolo tehnologije delotokov omogočajo večjo kontrolo draga izdelava nam ni treba začeti iz nič nam ni treba začeti iz nič dragoceno pridobljeno znanje, dragi viri znanje lahko delimo znanje lahko delimo boljši izkoristek virov boljši izkoristek virov aplikacije so večinoma odvisne od posameznih sistemov prevzem delotoka ne glede na sistem na katerega se bo izvajal prevzem delotoka ne glede na sistem na katerega se bo izvajal uporabniške zahteve po večji prožnosti, prilagodljivosti višja stopnja abstrakcije višja stopnja abstrakcije hiter razvoj inovativnih in zmogljivih aplikacij hiter razvoj inovativnih in zmogljivih aplikacij zahteve po varnosti in sledenja izvornosti delotoki omogočajo povsem odprti pristop k izvornosti (angl. Open Provenance) delotoki omogočajo povsem odprti pristop k izvornosti (angl. Open Provenance)

38. 38 Q&As

39. 39 Distributed Audio Retrieval Medical & bioinformatics communities Private EDA Industrial Applications AEC Professionals Data Services Community Infrastructures Enabler (for system administrators) Application Usage Knowledge Generation Social Networking Knowledge Generation Community Taverna WF EnactorTriana WF Manager Domain Knowledge Infrastructures & environment metadata Service Utilities Registry Application Usage, Provenance & Trust Data Registry Metadata Bus Application (Workflow) Hosting Layer Other WF systems Collaboration Spaces Data and Software Security, ID management InfrastructuresServices (security, logging, monitoring etc.) (New) Distributed Applications Resources, Middleware Services and Infrastructures Layer User Interfaces / Domain oriented views on resources and applications Application Composition Modules (for application developers) Knowledge Layer

40. 40 Primer delotoka (glasba)