Introduction to Distributed Computing Infrastructures and the Catania Science Gateway Framework Roberto Barbera Univ. of Catania & INFN Web course on the development of applications for Catania Science Gateways, 9 July 2013

2 Outline  Introductory concepts and driving considerations  The Catania Science Gateway Framework  Summary and conclusions

Time 3 Evolution of distributed computing Mainframe Computing 80’s-90’s Cluster Computing 90’s-00’s Grid Computing 00’s-10’s Cloud Computing Cost of hw Cost of networks Power of COTS WAN bandwidth

Research networks at «global» scale 4

The “Global” Grid

Elements of Distributed Computing Infrastructures

Job lifecycle

One architecture… many local resource managers 9

Genesis II The “non-global” middleware Interoperability and easiness of access are issues

11 Using Grids is not straightforward  Users have to cope with complex security procedures, execution scripts, job description languages, command line based interfaces and lack of standards This makes the learning curve very steep and keeps non IT-experts away

12 (1) The eResearch 2020 report (1) Some barriers in the adoption of Grids:  Changes on Grids means changes on applications  Time required to adapt usual workflows  Lack of structure to support anonymous access  Download and installation of applications  Interface  Slow to get to compared to other resources  Difficult to use in the beginning  Time spent to get the application compiled and running

13 Some numbers…

The cloud stack 14

A very «cloudy» sky… 15

… and difficult choices to make 16

17 The path to technology uptake – Where are we with e-Infrastructures ? The Rogers “bell-shape” curve - Rogers, E. M. (1962), “Diffusion of Innovations”, Glencoe: Free Press.

Interoperability – Does this definition apply to e-Infrastructures ?  Interoperability is a property referring to the ability of diverse systems and organizations to work together (inter-operate). The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to system performance;  According to ISO/IEC (Information Technology Vocabulary, Fundamental Terms), interoperability is "The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units". 18

19 Science Gateways “A Science Gateway is a community-developed set of tools, applications, and data that is integrated via a portal or a suite of applications, usually in a graphical user interface, that is further customized to meet the needs of a specific community.” Teragrid/XSEDE

IT acceptance model – the Grid Development of Science Gateways Requirement for sustainability Davis, F. D. (1989), "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly 13(3): 319–340 20

21 Building Science Gateways should be like playing with Sc. Gtwy E Sc. Gtwy DSc. Gtwy CSc. Gtwy B Sc. Gtwy A Standards Simplicity Easiness of use Re-usability

22 Summary of adopted standards  The framework for Science Gateways developed at Catania is fully web-based and adopts official worldwide standards and protocols, through their most common implementations  These are:  The JSR 168 and JSR 286 standards (also known as "portlet 1.0" and "portlet 2.0" standards)JSR 168JSR 286  The OASIS Security Assertion Markup Language (SAML) standard and its Shibboleth and SimpleSAMLphp implementationsOASISSecurity Assertion Markup LanguageShibboleth SimpleSAMLphp  The Lightweight Direct Access Protocol, and its OpenLDAP implementationOpenLDAP  The Cryptographic Token Interface Standard (PKCS#11) standard and its Cryptoki implementationCryptographic Token Interface Standard  The Open Grid Forum (OGF) Simple API for Grid Applications (SAGA) standard and its JSAGA implementationOpen Grid ForumSimple API for Grid Applications JSAGA

23 The Catania Science Gateway Framework ( Science Gateway Science Gateway App. 1 App. 2 App. N Embedded Applications Administrator Power User Basic User Users from different organisations having different roles and privileges Standard-based (SAGA) middleware-independent Grid Engine Standard-based (SAGA) middleware-independent Grid Engine Grid/Cloud/Local middleware supported so far

24 AuthN/AuthZ Schema AuthorisationAuthorisation Science Gateway GrIDP (“catch-all”) GrIDP (“catch-all”) IDPCT (“catch- all”) IDPCT (“catch- all”) IDP_y LDAP Register to a Service 2. Sign in Authentication Social Networks’ Bridge IdP

1. sign in User 6. get the results 4. execute action 3’/4’. track user eCSG Admin 5. get output Local/ Grid/ Cloud 2”. authZ eToken server 2’. authN Identity Provider User Registry Compliant with the EGI.eu Portal and Traceability Policies 3. create a proxy from an eToken server with robot certificates Catania Science Gateway Access Workflow

Robot certificates «in a nutshell» Robot certificates have been introduced to allow users, who do not have/want personal certificates and do not belong to any Virtual Organisation, to use the Grid They are usually issued on USB form factor’s smartcards Basically, these certificates can be used to identify a person responsible for an unattended service or process acting as client and/or server for a Virtual Research Community

The Catania Grid & Cloud Engine Grid/Cloud Engine Users Tracking DB Science GW Interface SAGA/JSAGA API Job Engine Data Engine Users Track & Monit. Science GW 1 Science GW 2 Science GW 3 Grid/Cloud/Local MWs Liferay Portlets eToken Server New ModifiedNewModified New

CHAIN Demo Contributors 28

CHAIN Demo Status ( 29

CHAIN Demo Status ( 30

31 The Africa Grid Science Gateway (

The GISELA Science Gateway (

The GISELA SG: A new concept of Application Registry Links to applications run pages

The GISELA Science Gateway: Phylogenetics (MrBayes) Input Parameters

Some considerations – Mobile Access  More than 25% of mobile phones in the world are smartphones and the number of people connected through mobile appliances increases every year  Social networking amounts to 91% of mobile internet access, compared to 79% on desktops, and it is expected that by 2014 mobile internet should take over desktop internet usage (*)  So, mobile access to “everything” is not any more an option; it is a must and e-Infrastructures shouldn’t/won’t be an exception (*)

36 “Mobile” Science Gateways

37 “Mobile” Science Gateways

38 “Mobile” Science Gateways

Big challenges are in front of us…  Now that many users can potentially access and use Catania Science Gateways, a new training and communication strategy is needed as well as a portfolio of appealing applications to attract them 39

Users/VRCs Ex.: Africa Grid Science Gateway SG Dev. The Science Gateway Market Place

Survey for VRCs to propose applications ( 41

Training for Science Gateway developers  New training material:  New training tools:  New training events (e.g., this course!):  

43 Summary and conclusions  e-Infrastructures can be very beneficial platforms for many users, provided they are really «easy to use» and users are at their centre  The Catania Science Gateway Framework changes the way Grid infrastructures are used, hugely widening their potential user base across continents and organisations, especially non-IT experts and the “citizen scientist”  The adoption of standards (JSR 286, SAGA, SAML, etc.) represents a concrete investment towards sustainability  We now need to establish a task force of Science Gateway developers able to quickly develop portlets for scientific applications to be included in the available Science Gateways  Developers will gain the access to a worldwide e-Infrastructure to develop and run their own applications

44 Thank you !