EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI user support & interaction with the VRCs Nuno Ferreira User Community Support Team EGI.eu WeNMR Devs Workshop, Utrecht, NL /13 1

EGI-InSPIRE RI The grid: a primer Managing a sustainable infrastructure Building sustainable user services Summary Outline WeNMR Devs Workshop, Utrecht, NL /13 2

EGI-InSPIRE RI What is the grid? “Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”. Foster, I. et al., Int. J. Superc. Appli. (2000)15:3 WeNMR Devs Workshop, Utrecht, NL /13 3 Source:

EGI-InSPIRE RI Infrastructure (Wikipedia) Infrastructure is the basic physical and organisational structures needed for the operation of a society or enterprise, or the services and facilities necessary for an economy to function European Grid Infrastructure (EGI) provides a service infrastructure that exposes and helps coordinate a resource infrastructure WeNMR Devs Workshop, Utrecht, NL /13 4 The Enterprise is the European Research Area

EGI-InSPIRE RI The grid: a primer Managing a sustainable infrastructure Building sustainable user services Summary Outline WeNMR Devs Workshop, Utrecht, NL /13 5

EGI-InSPIRE RI EGI Resource Infrastructure WeNMR Devs Workshop, Utrecht, NL /13 6 Resource Infrastructure Resource Centres Resource Infrastructure Resource Centres Resource Infrastructure Resource Centres Network Integrated Resource Infrastructure Provider National Grid Initiative Peer Resource Infrastructure Provider Integrated: infrastructure operated by a non-EGI-InSPIRE partner but relying on EGI operational services (MoU) Peer: infrastructure accessible to EGI users, but relying on own operational services MoUs EGI.eu

EGI-InSPIRE RI EGI Collaboration Context & NA2 - EGI-InSPIRE Review NGI: National Grid Initiative EIRO: European Intergovernmental Research Organisation NGI EIRO NGI EIRO Virtual Research Community NGI Virtual Research Community EGI.eu

EGI-InSPIRE RI Status and yearly increase WeNMR Devs Workshop, Utrecht, NL /13 8 Resource Centres 338 (+6.8%) : Europe, Asia Pacific, North and South America 96 supporting MPI (+31.5%) Countries 51 (57 with integrated RPs) (+18.75%) Capacity 240,000 CPU cores 339,000 with integrated and peer infrastructures (24.9%) 1.89 Million HEP-SPEC 06* 102 PB disk, 89 PB tape * HEP-SPEC 06: Computing benchmark based on SPECCPU2006, 10 HEP-SPEC = 4 kSI2k

EGI-InSPIRE RI EGI.eu Coordination for European Grid resources –Established February 8th 2010 –Central policy & services needed to run a grid –Sustainable small coordinating organisation Based in Amsterdam –Coordinating core (~20 people) in Amsterdam –Technical services from partners (~20 people) WeNMR Devs Workshop, Utrecht, NL /13 9 EGI and EGI.eu supported by EGI-InSPIRE project

EGI-InSPIRE RI EGI-InSPIRE project Integrated Sustainable Pan-European Infrastructure for Researchers in Europe A 4 year project with €25M EC contribution –Project cost €72M –Total Effort ~€330M –Effort: 9261PMs Project Partners (50) EGI.eu, 38 NGIs, 2 EIROs Asia Pacific (9 partners) Un-Funded WeNMR Devs Workshop, Utrecht, NL /13 10 Funded

EGI-InSPIRE RI EGI-InSPIRE objectives A sustainable production infrastructure –Resource providers in Europe and worldwide –With new technologies as they mature Support structured international research –Sustain current domain specific services –Attract new user communities (e.g. ESFRI) WeNMR Devs Workshop, Utrecht, NL /13 11

EGI-InSPIRE RI The grid: a primer Managing a sustainable infrastructure Building sustainable user services Summary Outline WeNMR Devs Workshop, Utrecht, NL /13 12

EGI-InSPIRE RI A virtuous service cycle WeNMR Devs Workshop, Utrecht, NL /13 Used by Researchers Gathering New Requirements New Technology Assessed Deployed Infrastructure Services 13 EGI.eu National Resource Providers Current User Communities New User Communities MoUs MoUs & OLAs Technology Providers MoUs & SLAs

EGI-InSPIRE RI UCB and the VRC model WeNMR Devs Workshop, Utrecht, NL /13 14 Benefits: 1.Defined communication channels: named contact points between communities & EGI 2.Application integration 3.Policy and procedures: define interaction between EGI and users 4.Requirements gathering: solutions and influence 5.Dissemination: sharing of news, breakthroughs, best practice and innovation User Community Board (UCB) with representation from: Life Sciences, WLCG, WeNMR, Astronomy & Astrophysics, Earth Sciences, Computational Chemistry, Arts & Humanities, Hydrometeorology VO: a group of researchers working collaboratively EGI Helpdesk Other Helpdesk E G I. e u NGINGI Training Events Trainers Technical Services VRC support units Other support units UCB User Community VRC: a grouping of research collaborations working in a common domain

EGI-InSPIRE RI NGI User Support Teams WeNMR Devs Workshop, Utrecht, NL /13 15 NGI User Support Teams Consultancy Training Provide access to grid and software services Porting apps. Dev. and operate software services Collecting Feedback Document Processes & apps. Helpdesk NGI User Support Teams provide: Consultancy & training Access to grid services Application porting Developing and operating software services Collecting feedback Documentation & helpdesk NGIs receive: Web & wiki resources Requirements process Applications and services VRC coordination support Support teams are available across 50 partners – they are all different… as are their support skills.

EGI-InSPIRE RI NGI User Support Teams WeNMR Devs Workshop, Utrecht, NL /13 16 NGI User Support Teams Consultancy Training Provide access to grid and software services Porting apps. Dev. and operate software services Collecting Feedback Document Processes & apps. Helpdesk NGI User Support Teams provide: Consultancy & training Access to grid services Application porting Developing and operating software services Collecting feedback Documentation & helpdesk NGIs receive: Web & wiki resources Requirements process Applications and services VRC coordination support Support teams are available across 50 partners – they are all different… as are their support skills.

EGI-InSPIRE RI Technical support services WeNMR Devs Workshop, Utrecht, NL /13 17 Services provided by a few NGIs for the whole community To support the Virtual Research Communities To facilitate the work of NGI support teams To improve communication among NGIs, among users and with users Bring in expertise and services from Heavy User Communities (HUCs) Tools and services (current list – will be expanded): 1.EGI Application Database (Marios Chatziangelou, GRNET) 2.Services for VOs (Ignacio Blanquer Espert, UPV ; Gonçalo Borges, LIP) 3.Training Marketplace (Claire Devereux, STFC) 4.Requirement Tracker (EGI.eu with help from CESNET)

EGI-InSPIRE RI EGI Application Database WeNMR Devs Workshop, Utrecht, NL /13 18 Benefits of AppDB: gives recognition to reusable applications and tools gives recognition to application developers Use of AppDB Register applications/tools Reuse of applications/tools Map of application landscape AppDB gadget has been instantiated by: –WeNMR, LSGC, GridPP, Serbian, UK and Hungarian NGIs

EGI-InSPIRE RI Coordination of services for VOs WeNMR Devs Workshop, Utrecht, NL /13 19 Coordination consists of: Reviewing available services: management, monitor & accounting tools (NAGIOS SAM, GANGA, DIANE, CERN dashboard & mini dashboard) Consultancy/help for VO managers (*) Specification of requirements for improving operational tools to simplify VO management Provisioning basic services for new VOs Developing dashboard infrastructure (*) Further consultancy is provided by NGIs NGI VO monitor NGI Site resources

EGI-InSPIRE RI VO Services: NAGIOS SAM WeNMR Devs Workshop, Utrecht, NL /13 20 Service Status Details for: phys.vo.ibergrid.eu NAGIOS SAM triggers the executions of probes in grid sites Tests to the Storage resources Tests to the Computing Resources Summary views of successful and unsuccessful services Sends s to the site admins in case of failures Provision of VO support software for VRCs & VOs (e.g. WeNMR, BioMed) Benefits: Sites from a VO are tested (infrastructure tests run probes on NGIs) Ability to develop and integrate VO’s own probes A VO can use the central installation from LIP/UPV, or can install its own

EGI-InSPIRE RI VO Services: VO Dashboard WeNMR Devs Workshop, Utrecht, NL /

EGI-InSPIRE RI Training marketplace Integration of: –Training event calendar –Digital library (of training materials) –Service offering & requesting form Purpose of Training Marketplace: –Register training events –Use, reuse and share training materials –Offer training-related services –Feed training-requirements –Delegate representative to EGI Training Working Group Results: –Approximately 60 training events were added during Project Year 1. WeNMR Devs Workshop, Utrecht, NL /

EGI-InSPIRE RI Processing EGI requirements WeNMR Devs Workshop, Utrecht, NL /13 23 TCB EGI Requirements Tracker VRCs Input Channels for User requirements NGIs HUCs ESFRIs EEF VOs Developer teams in TNA3.4 UCST (with NGIs) UCB External technology providers EIROs EGI Helpdesk (GGUS system) EMI IGE SAGA USAG Permanent storage for requirements Complete transparency –For user, NGIs & technology providers Requirement submission interface

EGI-InSPIRE RI WeNMR RT Dashboard WeNMR Devs Workshop, Utrecht, NL /13 24

EGI-InSPIRE RI PY1 Results: Requirements Tracker 110 new requirements: –48 resolved within EGI-InSPIRE/UCST –36 prioritised by UCB of which… –20 passed via TCB to Technology Providers –26 still being investigated by UCST Current issues endorsed by UCB and recognised by TCB: 1.Increased stability and scalability for gLite WMS 2.Better (more verbose and informative) error messages 3.Fixing the known bugs before adding new features 4.Coherency of command line commands, parameters and APIs 5.Better feedback about jobs, automated resubmission of jobs stuck on sites Requirements to be refined in community workshops Data management, Workflow systems & Workload management Use to define and build EGI VRC User support platform WeNMR Devs Workshop, Utrecht, NL /13 25 Proved to be an effective communication channel with EMI and other external Technology Providers 60 requirements also generated from legacy documents from VRCs: SHARE Roadmap (HealthGrid), ESFRI project reqs., Production Grid Infrastucture (OGF), Request for Enhancements of gLite to support bio-NMR applications, gLite requirements (EGEE), User requirements (EGI-DS), DRIHMS Questionnaire

EGI-InSPIRE RI Community Support Workshops Motivation for workshops (from EGI Forums): –Users indicated need for – collaboration & exploration –More smaller and focused meetings needed –Bring specialists together Purpose: –Inform and educate, clarify and specify Emerging themes (from requirements): –portals, visualisation, data management, scalability, workflow, supporting users, planning training events, developing and provisioning sustainable apps. WeNMR Devs Workshop, Utrecht, NL /

EGI-InSPIRE RI WeNMR Devs Workshop, Utrecht, NL /13 27

EGI-InSPIRE RI WeNMR as a VRC WeNMR signed an MoU with EGI 11 April 2011 WeNMR Devs Workshop, Utrecht, NL /

EGI-InSPIRE RI Summary EGI VRC model available and WeNMR VRC created –WeNMR participating in User Community Board –WeNMR contributing to Requirements Tracker process –EGI supporting NGIs who support WeNMR Technical Support Services: User requirements AppDB gadget VO services Training Marketplace Community Support Workshops: –Themes under investigation WeNMR Devs Workshop, Utrecht, NL /13 29