1. Advantages of adopting late-binding techniques through standardised interfaces for workflow managers. A.J. Rubio-Montero 1, M. Plociennik 2, I. Marín-Carrión 3, T. Zok 2, M.A. Rodríguez-Pascual 1, R. Mayo-García 1, M. Owsiak 2, E. Huedo 3, F. Castejón 1,4 and B. Palak 2. 1 CIEMAT, Centro de Investigaciones Energéticas Medioambientales y Tecnológicas, Avda. Complutense, 40, 28040 Madrid, Spain. 2 PSNC, Poznań Supercomputing and Networking Center, IBCh, PAS, Noskowskiego 12/14, 61-704 Poznań, Poland. 3 Fac. Informática, UCM, Prof. José García Santesmases s/n. 28040 MADRID, Spain. 4 Euratom-CIEMAT Association, Avda. Complutense, 40, 28040 Madrid, Spain. This work is the result of a coordinate effort supported by the EGI SA1, SA3 and IGE WP4 Tasks. [1] I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludaescher and S. Mock. Kepler: An Extensible System for Design and Execution of Scientific Workflows, Proc. of SSDBM 2004. pp. 423–424. DOI:10.1109/SSDM.2004.1311241 [2] L. Cabellos, I. Campos, E. Fernández-del-Castillo, M. Owsiak, B. Palak and M. Plociennik. Scientific workflow orchestration interoperating HTC and HPC resources, Comp. Phys. Comm. 182(4) (2011) 890–897. DOI: 10.1016/j.cpc.2010.12.020 [3] G. Juve, E. Deelman, K. Vahi, G. Mehta, Experiences with resource pro-visioning for scientific workflows using Corral, Scientific Programming18 (2) (2010) 77–92. DOI: 10.3233/SPR-2010-0300 [4] P. Kacsuk,T. Kiss, G. Sipos. Solving the grid interoperability problem by P-GRADE portal at workflow level, Future Generation Computer Systems 24(7) (2008) 744–751 DOI: 10.1016/j.future.2008.02.008 [5] E. Huedo, R. S. Montero, I. M. Llorente, A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services, Future Generation Computer Systems 23 (2) (2007) 252–261. DOI: 10.1016/j.future.2006.07.013 [6] A.J Rubio-Montero, E. Huedo, F. Castejón, R. Mayo-García. GWpilot: Enabling multi-level scheduling in distributed infrastructures with GridWay and pilot jobs. To appear. [7] I. Marín, E. Huedo, I. M. Llorente. Interoperating Grid Infrastructures with the GridWay Metascheduler, Concurrency and Computation: Practice and Experience. (2012). In press. DOI: 10.1002/cpe.2971 Objectives To take advantage of the GridWay modularity to build a new multi-level scheduling framework based on the performance and reliability provided by Gwpilot. To achieve portability and interoperability required by EGI and OGF through the OGSA-BES interface. To preserve backward compatibility with legacy middleware by means of Kepler actors and GridWay drivers. Objectives To take advantage of the GridWay modularity to build a new multi-level scheduling framework based on the performance and reliability provided by Gwpilot. To achieve portability and interoperability required by EGI and OGF through the OGSA-BES interface. To preserve backward compatibility with legacy middleware by means of Kepler actors and GridWay drivers. Introduction Visual workflow systems such as Kepler [1] are fundamental tools to implement and maintain complex scientific simulations. These workflow managers allow directly interfacing HPC platforms, databases, repositories or grid resources in a single framework [2]. However, this approach means to be tied to middleware implementations and to bear the performance slowdown due to waiting queues. A way to improve the grid execution performance is the use of pilot-job techniques, which were successfully tested with workflows managers in approaches as Pegasus [3] or P-Grade [4]. Nevertheless, they inherit the issues from current pilot systems that lack some features such as easy-installing, user-sharing or the absence of standardised interfaces to remotely access which make them unfeasible for many projects. The aim of this work is to offer an unified, simplified, efficient, flexible, standardised and portable new mechanism to remotely manage pilot jobs and the tasks that compound a Kepler workflow. For this purpose, the new pilot system framework GWpilot [5] based on the GridWay [6] meta-scheduler is utilised and accessed through the standard OGSA-BES interface [7]. Introduction Visual workflow systems such as Kepler [1] are fundamental tools to implement and maintain complex scientific simulations. These workflow managers allow directly interfacing HPC platforms, databases, repositories or grid resources in a single framework [2]. However, this approach means to be tied to middleware implementations and to bear the performance slowdown due to waiting queues. A way to improve the grid execution performance is the use of pilot-job techniques, which were successfully tested with workflows managers in approaches as Pegasus [3] or P-Grade [4]. Nevertheless, they inherit the issues from current pilot systems that lack some features such as easy-installing, user-sharing or the absence of standardised interfaces to remotely access which make them unfeasible for many projects. The aim of this work is to offer an unified, simplified, efficient, flexible, standardised and portable new mechanism to remotely manage pilot jobs and the tasks that compound a Kepler workflow. For this purpose, the new pilot system framework GWpilot [5] based on the GridWay [6] meta-scheduler is utilised and accessed through the standard OGSA-BES interface [7]. Advantages Standardised remote task delegation through OGSA. A Kepler simulation that can be check-pointed and carried out on laptops. Several Kepler instances (or other applications) can also use the platform. Underneath management of user tasks and pilot jobs without control loss of provisioning: Task requirements will automatically be translated to pilot provisioning. Two ways for data allocation: via customisation tasks or directly accessing to storage services. Pre-allocation, reservation, data-location awareness, and multi-user coordination are allowed. Advantages Standardised remote task delegation through OGSA. A Kepler simulation that can be check-pointed and carried out on laptops. Several Kepler instances (or other applications) can also use the platform. Underneath management of user tasks and pilot jobs without control loss of provisioning: Task requirements will automatically be translated to pilot provisioning. Two ways for data allocation: via customisation tasks or directly accessing to storage services. Pre-allocation, reservation, data-location awareness, and multi-user coordination are allowed. Benefits and impact The creation, maintenance and execution of workflows with pilots jobs are simplified. It allows implementing complex scheduling policies to improve specific workflows. Benefits and impact The creation, maintenance and execution of workflows with pilots jobs are simplified. It allows implementing complex scheduling policies to improve specific workflows. Technical Forum Madrid, 16 th -20 th Sept. 2013