E-science grid facility for Europe and Latin America EELA-2 Achievements and Impact so far Towards a Latin American Grid Initiative (LGI) Philippe Gavillet (Deputy Project Coordinator) on behalf of EELA-2 Contributions from B.Marechal (PC), R.Barbera (TC), F.Brasileiro (JRA1 AM)

Barcelona (Spain) – EGEE’09 – Outline EELA-2 in a nutshell State of the Art – A few highlights The EELA-2 model for long-term sustainability Outlook

Barcelona (Spain) – EGEE’09 – EELA-2 objectives Provide an empowered Grid Facility with versatile services fulfilling application requirements Ensure production quality services Expand the current EELA e-Infrastructure Look for new communities Ensure the long term sustainability of the e- Infrastructure beyond the term of the project

Barcelona (Spain) – EGEE’09 – Project in numbers EELA-2 (CP-CSA under FP7) E-science grid facility for Europe and Latin America EC support: 2.1 M€ CETA-CIEMAT extra funds: 0.3 M€ Currently 16 Countries (11 in LA) 2 new countries (Panama & Uruguay) 1 International Organisation (CLARA) Currently 78 Members (62 in LA) 32 Institutions joining (31 in LA) Currently 14 JRUs (9 in LA) 4 new JRUs (3 in LA)

Barcelona (Spain) – EGEE’09 – Countries and Resources Centres

Barcelona (Spain) – EGEE’09 – NA2: Dissemination & Training (1/2) Much more training effort delivered than pledged

Barcelona (Spain) – EGEE’09 – NA2: Dissemination & Training (2/2) Status vs Roadmap 1

Barcelona (Spain) – EGEE’09 – NA3: Application Support (1/3) The EELA-2 “gridification universe”

Barcelona (Spain) – EGEE’09 – NA3: Application Support (2/3) 27 apps using the EELA-2 Grid facility 19 papers published

Barcelona (Spain) – EGEE’09 – NA3: Application Support (3/3) Some examples… Water control NMR tomographyFire growth simulationHeart simulator

Barcelona (Spain) – EGEE’09 – “Expanding the current e-Infrastructure to consist of 41 Resource Centres (13 in Europe and 28 in Latin America) mobilising about 3000 computing cores and more than 700 Terabytes of storage space, at start of the project and to further grow by 20 % in computing and 15 % in storage over the duration of the project.” “Providing the full set of Infrastructure and Application- oriented Grid Services needed by all types of applications in their scientific environment.” The current infrastructure has 22 Resource Centres (11 in Europe and 11 in Latin America) mobilising about 5800 computing cores and more than 192 Terabytes of storage. Two Core Service Centres deployed (EU and LA), providing a full set of Grid Services (information systems, data and metadata catalogues, etc.) and the services developed by JRA1. SA1: Grid Infrastructure Service (1/3)

Barcelona (Spain) – EGEE’09 – “Building the support of the e-Infrastructure to provide high reliability services to partnerships spanning the Atlantic Ocean by:” –“Providing a complete set of Global Services from central EELA-2 Operation and Networking Support Centres (EOC/ENSC).” –“Targeting the creation of Regional Operation Centres (ROC) in Latin America, covering one or more countries.” –“Pursuing the aggressive EELA Training and Dissemination programme.” –“Creating a comprehensive knowledge repository on Grid operation federated with the EGEE ones.” A full set of ancillary tools was deployed on the EOC, which acts as a central co-ordination point for the infrastructure. Four OOD teams were engaged in Operations; they prefigure the support teams of future Latin American ROCs. SA1 contributed trainers and infrastructure support to NA2 and NA3. As first step, SA1 and EGEE-SA1 agreed on common protocols and procedures. OOD Teams SA1: Grid Infrastructure Service (2/3)

Barcelona (Spain) – EGEE’09 – Resource Centres Map (Y1) SA1: Grid Infrastructure Service (3/3)

Barcelona (Spain) – EGEE’09 – SA2: Network Resource Provision (1/2) Integration of Network Support in EOC

Barcelona (Spain) – EGEE’09 – SA2: Network Resource Provision (2/2) EELA-2 Network Monitoring

Barcelona (Spain) – EGEE’09 – JRA1: Application Services (1/3) EELA-2 Software Repository Data Management –Digital archives –Secure storage –Cooperative annotation of data Job management –VO compliant Virtual Machine environments –Workflow for OurGrid jobs –WatchDog (monitoring and control of job execution on the gLite Worker Node) Catalogue and file management –lcg-rec toolkit (recursive version of the lgc-* suite of commands)

Barcelona (Spain) – EGEE’09 – JRA1: Infrastructure Services (2/3) X.509-enabled OurGrid middleware available –Enables an OurGrid system to use the same PKI and Grid Security Infrastructure technologies used by gLite systems Interoperability between OurGrid and gLite planned for M16 is advancing very well –MoU signed with other projects working on similar goals (EDGeS and SEE- GRID-SCI) Port of gLite to Microsoft Windows platforms –User Interface:  Workload Mngt, File transfer, Data Mngt, Info System Support for grid operation –SAGE (Storage Accounting for Grid Environments) –Multiple middleware Resource Centres  gLite and Globus Toolkit  gLite and OurGrid

Barcelona (Spain) – EGEE’09 – JRA1: Use of new Services (3/3) A number of applications are either using or being upgraded to use the new Services ServiceApplications GSAFAîuri – UFRJ (Brazil) OurGridDistBlast - UnB (Brazil), BRAMS - UFCG (Brazil) and Cinefilia - INFN (Italy) watchdogCROSS-Fire – U. of Minho (Portugal) lcg-rec toolkit CROSS-Fire – U. of Minho (Portugal) ServiceApplications GSAFgRREEMM - Cubaenergia (Cuba) Secure Storage CardioGrid Portal - UBA (Argentina) watchdogG-InterproScan and G-HMMER - UNIANDES (Colombia) lcg-rec toolkit META-Dock - UNAM (Mexico) Using Being upgraded

Barcelona (Spain) – EGEE’09 – Next to EELA-2….

Barcelona (Spain) – EGEE’09 – The “global” Grid coverage

Barcelona (Spain) – EGEE’09 – The Latin American Grid Initiative (LGI) ( GridLGI is composed of three layers, each of them including Grid and Network components: At the lower Institute level: –The Resource Centre (RC) is the actual site of the Grid computing resources; –The backbone LAN or MAN network connects the Computing and Storage Elements (CE & SE) at the RC site; At the intermediate national level: –The National Grid Initiative (NGI) integrates all Grid Institutions in the country and federates under its Grid Operation Centre (GOC) all their RCs in a National Grid (GridNGI); –The underlying National Research & Education Network (NREN) links country-wide the Research and Education Institutes and optimizes / monitors their network activity via its associated Network Operation Centre (NOC); At the upper international layer: –The Latin American Grid Initiative (LGI) which brings together all NGIs and supervises the various GridNGIs operation from its Grid Support Centre (GSC); –CLARA: it is the WAN network which integrates the Latin American NRENs and supervises the overall RedCLARA network activity by means of its Network Support Centre (NSC).

Barcelona (Spain) – EGEE’09 – RC tasks

Barcelona (Spain) – EGEE’09 – RC tasks (detailed description) Deploy and operate the local grid services such as Computing Elements and Storage Elements Provide 1st level support to users, application developers and Resource Centre administrators, usually through a trouble- shooting ticketing system Coordinate the deployment and update of the grid middleware used by its resource providers Monitor the status of the grid services run by the national resource providers to quickly detect problems, request of corrections and be sure that problems are fixed Deal with security incidents making sure that proper solutions are quickly applied Deploy an authentication service; this includes one or more Registration Authorities of the National Certification Authority Operate an accounting repository to keep track of grid resource usage.

Barcelona (Spain) – EGEE’09 – GOC tasks

Barcelona (Spain) – EGEE’09 – GOC tasks (detailed description) Deploy and operate the core grid services needed to interconnect to the GSC. It includes Workload Management Systems, Grid Information Systems, Data Management Systems, etc. Provide 2nd level support to users, application developers and resource centre administrators in the region Setup, support the Operator on Duty (OOD) Teams responsible for the continuous monitoring and support of the Resource Centre fabric, for prompt error detection, diagnosis and resolution of operational problems Route and follow up trouble-shooting tickets to the proper support unit Coordinate the deployment and updating of the grid middleware Monitor the status of the grid services running in the country to quickly detect problems, request corrections and be sure that problems are fixed Coordinate the solution of security incidents, making sure that proper solutions are quickly applied Operate an accounting repository to keep track of grid resource usage; Operate a configuration repository Operate the VO management systems for country-wide-scope Virtual Organisation Coordinate the resource allocation and brokering for national VOs Deploy and operate an Authentication Service at national level. This includes a Certification Authority and as many Registration Authorities as needed

Barcelona (Spain) – EGEE’09 – GSC tasks

Barcelona (Spain) – EGEE’09 – GSC tasks (detailed description – 1/2) Coordinate the operation of all GOCs Deploy and operate the core grid services needed to interconnect all the GOCs and the single RCs in the infrastructure. It is usually done in an hierarchical way, profiting from the services already deployed and maintained by the GOCs, including Workload Management Systems, Grid Information Systems, Data Management Systems, etc. Provide 3rd level support to users, application developers and Resource Centre administrators in the infrastructure Route and follow up trouble tickets with the GOCs Maintain and Operate a central ticketing system Maintain and Operate a central Operations Portal Maintain a central Knowledge Database Maintain a Knowledge Sharing System Define best practices, operation procedures and operation requirements Coordinate the middleware deployment, release and support

Barcelona (Spain) – EGEE’09 – GSC tasks (detailed description – 2/2) Gather requirements for new services from users, resource centre administrators and grid operators Advertise new Infrastructure and Application services to users and applications developers Deploy and operate an Authentication Service. This includes a “catch-all” Certification Authority having as many Registration Authorities as needed in those countries which do not yet have a national Certification Authority; the GSC should also provides consulting services to those countries in order to speed-up the creation of new Certification Authorities and their accreditation by TAGPMA, the relevant Policy Management Authority Operate the VO management for global VOs Coordinate the resource allocation and brokering for global VOs Coordinate the solution of security incidents, making sure that proper solutions are quickly applied Operate a global accounting repository to keep track of grid resource usage Operate a global configuration repository Coordinate interoperation efforts with other grid infrastructures Perform studies to plan the growth of the infrastructure

Barcelona (Spain) – EGEE’09 – NSC tasks

Barcelona (Spain) – EGEE’09 – NSC tasks (detailed description) Be a single point of contact for network related issues Trace and diagnose network problems –The NSC could receive two types of tickets:  User ticket: an e-Infrastructure user notifying about a network issue on his/her site or within some site that he/she is trying to access  NREN ticket: a NREN notifying the NSC about network issues. It is important to note the assumption that the NREN is not expected to notify (at least initially) the NSC only about network issues which involve the e Infrastructure sites, but actually about all the NREN issues, e.g., through a general mailing list assignment Support E2E communication between sites Provide quality indicators about the network infrastructure

Barcelona (Spain) – EGEE’09 – Training & dissemination tasks

Barcelona (Spain) – EGEE’09 – Training & dissemination tasks (detailed description – 1/2) The activities to be performed by the tutors operating at NGI and RC level include: –Participate as instructors in tutorials at local, national levels in cooperation with the LGI –Update their knowledge as improvements and new services are added to the middleware(s) –Create, update and translate training material maintained by the LGI The dissemination activities include: –Coordinate the dissemination actions at local and national levels in cooperation with the LGI –Produce press releases, brochures, flyers, posters, multimedia material, and keep updated the contents (related to advancements plus news and reports) of websites in order to inform appropriately about the technology and events organized by the LGI as well as related communities –Organise workshops and conferences at local and national levels in cooperation with the LGI

Barcelona (Spain) – EGEE’09 – Training & dissemination tasks (detailed description – 2/2) The activities at the LGI level can be summarized as follows: –Definition of training plans. At least 2 travels per country and per year should be foreseen in order to support and define common training strategies with all participating countries –Designing and maintaining a master event calendar scheduling the most important LGI training and dissemination events like Grid schools, conferences and workshops –Maintenance of a central training material repository; training material should also be translated from English to Spanish and Portuguese by this task; the maintenance of a training database will be also provided by this task –Deliver Training courses for trainers –Offer consultancy to Application Developers when needed by the Application support Team.

Barcelona (Spain) – EGEE’09 – User support tasks

Barcelona (Spain) – EGEE’09 – User support tasks (detailed description) The activities that must be performed by the Application Support teams located either at the GOCs at the GSC include: –Answer tickets issued by the central support ticket system –Participate in periodic virtual meetings with application developers and forward their request if necessary to the middleware developers –Contribute for the creation of FAQs and “gridification” guidelines –Visit or accepting visits of application experts in events similar to the current EELA-2 "Gridification Weeks” –Participate as a tutor in the training events organized by the LGI –Make available a public central database containing the profile and the current status of all applications supported by the GridLGI –Act as a liaison between the GridLGI and new application communities or related projects –Evaluate the technical requirements and the impact of new projects/applications willing to be supported by the GridLGI

Barcelona (Spain) – EGEE’09 – NGI / LGI Management-Governance A tight liaison with the NREN is strongly suggested/recommended. Local conditions permitting, NGI should be incorporated into the NREN

Barcelona (Spain) – EGEE’09 – Preparing the “After EELA-2” (1/4) M06: Definition of the Strategy –WNA1.1: “Strategy towards Long Term Sustainability” Four lines of actions carried on in parallel Establish the conditions of a sustainable Latin American Grid Infrastructure –M06: DSA1.3: “The Long-Term Latin American Grid Initiative: Model & Operation Cost Estimate” –M16: WNA1.2: “Time Profiles for the Staffing of the Latin American Grid Initiative (LGI)”

Barcelona (Spain) – EGEE’09 – Preparing the “After EELA-2” (2/4) Collaboration with Grid-related projects –Pro-active participation in all Concertation meetings with other EGEE- related projects at EGEE’07, EGEE’08, ICT’08 (BELIEF Workshop) –Active contribution to the CHAIN preparation phase  Unfortunately we could not fit well our mandatory Services: SA1, SA2, JRA1 activities in the proposal (CSA-type)  Decision to go for a CP-CSA proposal to achieve our main objective: Garantee the sustainability of the LA part of the e_Infrastructure  CIEMAT in CHAIN leadering WP3: ”Present and emerging needs of trans- continental scientific communities” addressing VRC support  Wish to collaborate on VRC support with CHAIN, EGI,.. Common Application database with inputs from regions User guide, self-training material, Help desk,..

Barcelona (Spain) – EGEE’09 – Preparing the “After EELA-2” (3/4) Interaction with / Support from User Communities –Over Year-1: New collaborators joining EELA-2  2 Partners (Panama, Uruguay)  36 New Members (> 15 Applications) –JRU / NGI status  In place in: Argentina, Brazil, Chile, Colombia, Ecuador, France, Ireland, Italy, Mexico, Panama, Peru, Portugal, Spain, Venezuela  Missing: Cuba –Initiative with eResearch 2020 and Belief of an “e-Infrastructure Survey”

Barcelona (Spain) – EGEE’09 – Preparing the “After EELA-2” (4/4) Interaction with / Support from Decision Makers Actions –Decision Makers interactions already made:  Panama, Oct. 08  Lima, Nov. 08  Monterrey (Mexico), Oct. 08  Quito (Ecuador), Dec. 08  Bogotá (Colombia), Feb. 09 –In depth discussions (EELA-2 presentations, e-Science status in the country, e-Science Virtual days, prospective,..) –Production of dissemination materials (Posters, Flyers,..)

Barcelona (Spain) – EGEE’09 – Summary & Conclusions EELA-2 is well on track to achieve its objectives; EELA-2 has devised a comprehensive model and architecture of the Latin American Grid Initiative; The collaboration with CLARA has been settled and is becoming effective; EELA-2 has developed the necessary synergies with the other Grid projects, User Communities, Decision Makers; However, the roof will be solid only if the pillars will be strong so Latin American countries should proceed with the creation of NGIs in close relationships with NRENs.

Barcelona (Spain) – EGEE’09 – Thank you very much! …Any Questions ?