A sustainable e-Infrastructure for Europe Barcelona, Spain 28 March 2006 Bob Jones.

Slides:



Advertisements
Similar presentations
How to form a consortium
Advertisements

Research Infrastructures for e-Science
From CESSDA to European Research Infrastructure Developments in cross-European data sharing.
The EGI – a sustainable European grid infrastructure Michael Wilson STFC RAL.
We’re here for you. “European Exchange of Best Practice in Arson Investigation and Prevention” European exchange of best practice in arson investigation.
23 June Strategy Proposal Heinz Stockinger on behalf of the Executive Board SwiNG Assembly Meeting Berne, 23 June 2008.
Research and Innovation Research and Innovation Research and Innovation Research and Innovation Research Infrastructures and Horizon 2020 The EU Framework.
Delegations IV KAM Prague 3rd to 7th September 2014.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE-III Program of Work Erwin Laure EGEE-II / EGEE-III Transition Meeting CERN,
European Life Sciences Infrastructure for Biological Information ELIXIR
1 Introduction to EGEE-II Antonio Fuentes Tutorial Grid Madrid, May 2007 RedIRIS/Red.es (Slices of Bob Jone, Director of EGEE-II.
A sustainable e-Infrastructure for Europe Athens, Greece 19 April 2006 Bob Jones.
EPOS Preparatory phase Torild van Eck (ORFEUS) Call INFRA Deadline: December 3, 2009 Funding: between 3 and 6 MEuro Duration: max 4 year.
Enabling Grids for E-sciencE EGEE III Security Training and Dissemination Mingchao Ma, STFC – RAL, UK OSCT Barcelona 2009.
INFSO-RI Enabling Grids for E-sciencE SA1: Cookbook (DSA1.7) Ian Bird CERN 18 January 2006.
SEE-GRID-2 The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no
Towards the definition of an eIRGRoma, 10 December An e-Infrastructure in Europe: a strategy and policy driven approach for a policy eIRG A pink.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Related Projects Dieter Kranzlmüller Deputy.
Results of the HPC in Europe Taskforce (HET) e-IRG Workshop Kimmo Koski CSC – The Finnish IT Center for Science April 19 th, 2007.
Rackspace Analyst Event Tim Bell
Notur: - Grant f.o.m is 16.5 Mkr (was 21.7 Mkr) - No guarantees that funding will increase in Same level of operations maintained.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks ?? Athens, May 5-6th 2009 Community Support.
Responsibilities of ROC and CIC in EGEE infrastructure A.Kryukov, SINP MSU, CIC Manager Yu.Lazin, IHEP, ROC Manager
EGEE-II INFSO-RI Enabling Grids for E-sciencE Introduction EGEE/EGEE-II transition meeting – day 2 Bob Jones, EGEE-II Project Director.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE – paving the way for a sustainable infrastructure.
INFSO-RI Enabling Grids for E-sciencE Plan until the end of the project and beyond, sustainability plans Dieter Kranzlmüller Deputy.
Enabling Grids for E-sciencE 1 EGEE III Project Prof.dr. Doina Banciu ICI Bucharest GRID activities in RoGrid Consortium.
Bob Jones Technical Director CERN - August 2003 EGEE is proposed as a project to be funded by the European Union under contract IST
INFSO-RI Enabling Grids for E-sciencE NA5 – Policy and International Cooperation Panagiotis Louridas, Fotis Karagiannis, GRNET Final.
EIRG – e-Infrastructure Reflection Group Dieter Kranzlmüller e-IRG Chairman
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE II: an eInfrastructure for Europe and.
INFSO-RI Enabling Grids for E-sciencE External Projects Integration Summary – Trigger for Open Discussion Fotis Karayannis, Joanne.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Security Coordination Group Linda Cornwall CCLRC (RAL) FP6 Security workshop.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SA1: Grid Operations Maite Barroso (CERN)
INFSO-RI Enabling Grids for E-sciencE EGEE SA1 in EGEE-II – Overview Ian Bird IT Department CERN, Switzerland EGEE.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Security Coordination Group Dr Linda Cornwall CCLRC (RAL) FP6 Security workshop.
EGEE-III-INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE-III All Activity Meeting Brussels,
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The EGEE User Support Infrastructure Torsten.
EPOS where we are, where we go ! Massimo Cocco INGV Rome September EPOS Meeting.
Dieter European Grid Initiative Towards a sustainable production grid infrastructure.
© Enterprise Europe Network South West 2009 The Eurostars Programme Kenny Legg R&D Funding for the Environmental Sector – 29 June 2010 European Commission.
EGI-InSPIRE Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III 1GDB - December 2009.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks South-West Federation Gabriel Amorós (CSIC)
INFSO-RI Enabling Grids for E-sciencE An overview of EGEE operations & support procedures Jules Wolfrat SARA.
INFSO-RI Enabling Grids for E-sciencE Policy and International cooperation Fotis Karayannis, NA5 activity leader All Activity Meeting.
WLCG Laura Perini1 EGI Operation Scenarios Introduction to panel discussion.
SEE-GRID-2 The SEE-GRID-2 initiative is co-funded by the European Commission under the FP6 Research Infrastructures contract no
EGEE is a project funded by the European Union under contract IST EGEE Summary NA2 Partners April
EGEE is a project funded by the European Union under contract IST Roles & Responsibilities Ian Bird SA1 Manager Cork Meeting, April 2004.
Grupo de Redes y Computación de Altas Prestaciones Vicente Hernández Universidad Politécnica de Valencia Primera Reunión sobre e-Ciencia Andaluza (proyecto.
25-September-2005 Manjit Dosanjh Welcome to CERN International Workshop on African Research & Education Networking September ITU, UNU and CERN.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NA5: Policy and International Cooperation.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NGI Policy Workshop Introduction and scope.
E-science grid facility for Europe and Latin America CHAIN Proposal v0.1 EELA-2 compilation CERN e-Infrastructure projects Meeting ( )
Representation of European Grid efforts, international cooperation, and ESFRI EGI_DS WP3 consolidation workshop, CERN Fotis Karayannis, GRNET.
Introduction to the OECD. 4 key questions Who are we? What do we do? How do we do it? What happens next?
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NA5: Policy and International Cooperation.
Setting up NGI operations Ron Trompert EGI-InSPIRE – ROD teams workshop1.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NA5: Policy and International Cooperation.
INFSO-RI Enabling Grids for E-sciencE EGEE general project update Fotis Karayannis EGEE South East Europe Project Management Board.
Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education Data Sharing Grant.
INFSO-RI Enabling Grids for E-sciencE EGO Workshop Straw-man model 30 – 31 January 2006 Bob Jones, CERN.
INFSO-RI Enabling Grids for E-sciencE NA5 – Policy and International Cooperation Panagiotis Louridas, Fotis Karagiannis, GRNET Final.
Bob Jones EGEE Technical Director
LCG Security Status and Issues
Ian Bird GDB Meeting CERN 9 September 2003
Luděk Matyska CESNET ERIC Survey Results Luděk Matyska CESNET
Long-term Grid Sustainability
EGEE support for HEP and other applications
Connecting the European Grid Infrastructure to Research Communities
Presentation transcript:

A sustainable e-Infrastructure for Europe Barcelona, Spain 28 March 2006 Bob Jones

Barcelona, 28 March Contents The vision of an e-Infrastructure for Europe The international perspective Proposing a possible model –Mission –Structure –Key services –Funding & Governance Discussion Additional material – details of key services

Barcelona, 28 March e-Infrastructure for Europe The Vision (1) –“An environment where research resources (H/W, S/W & content) can be readily shared and accessed wherever this is necessary to promote better and more effective research” (1) “A European vision for a Universal e-Infrastructure for Research” by Malcolm Read irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdfhttp:// irg.org/meetings/2005-UK/A_European_vision_for_a_Universal_e-Infrastructure_for_Research.pdf

Barcelona, 28 March International Perspective Recommendations (1) of the Global Science Forum (2) of the Organisation for Economic Co-operation and Development 1: “Grids deserve to be treated as research infrastructures in their own right….. Governments should also consider taking steps to strengthen the international mechanisms for co-operation and co-ordination at the scientific, commercial and intergovernmental levels” 2: “A number of important Grid-related issues need to be addressed in a strategic, systematic, organised, international manner, with the participation of all the stakeholders: funding agency officials, Grid architects, implementers, and users from the scientific, academic and industrial communities” 3: “Consideration should be given to the creation of new mechanisms (or the strengthening of existing ones) to facilitate access to Grids for researchers and research organisations in developing countries, plus other appropriate measures to broaden international participation in grid projects” (1) Report on Grids and Basic research Programmes, Sydney September , (2) Australia, Austria, Belgium, Denmark, France, Germany, Italy, Japan, Netherlands, New Zealand, Sweden, United Kingdom, United States

Barcelona, 28 March The view of the e-IRG e-IRG Recommendation: “The e-IRG recognises that the current project-based financing model of grids (e.g., EGEE, DEISA) presents continuity and interoperability problems, and that new financing and governance models need to be explored – taking into account the role of national grid initiatives as recommended in the Luxembourg e-IRG meeting.” White Paper: e-IRG Open Workshop: April 10-11, 2006, Linz, Austria  Towards Sustainable e-Infrastructures

Barcelona, 28 March Proposed model of a e-Infrastructure for Europe This model builds on the experience gained with the EGEE project, infrastructure and applications Takes into account the OECD & eIRG recommendations Input from EGO paper (1) and the workshops held in Thoiry (France) st January and Kassel (Germany) 10 th March (1) Establishing an European Grid Organisation (EGO),

Barcelona, 28 March Mission Infrastructure –Manage and operate production Grid for European Research Area –Interoperate with e-Infrastructure projects around the globe –Contribute to Grid standardisation and policy efforts Support applications from diverse communities –Astrophysics –Computational Chemistry –Earth Sciences –Finance –Fusion –Geophysics –High Energy Physics –Life Sciences –Multimedia –… Business –Forge links with the full spectrum of interested business partners Disseminate knowledge about the Grid through training

Barcelona, 28 March Structure Federated model bringing together National Grid Initiatives (NGIs) to build a European organisation EGEE federations would evolve into NGIs Each NGI is a national body Recognised at the national level Mobilises national funding and resources Contributes to international standards and policies Operates the national e-Infrastructure Application independent Supports existing and new communities

Barcelona, 28 March European National Grid Projects Austria – AustrianGrid Belgium – BEGrid Bulgaria – BgGrid Croatia – CRO-GRID Cyprus – CyGrid Czech Republic- METACentre Slovakia Denmark Finland France - planned Germany – D-GRID Greece - HellasGrid Hungary Ireland - Grid-Ireland Israel – Israel Academic Grid Italy - planned Netherlands – DutchGrid Norway – NorGird Poland Portugal - planned Romania – RoGrid Serbia – AEGIS Slovenia SiGNET Spain - planned Sweden – SweGrid Switzerland Turkey – TR-Grid United Kingdom - eScience Based on information gathered for EGEE-II proposal in September 2005

Barcelona, 28 March A global, federated e-Infrastructure EGEE infrastructure ~ 200 sites in 39 countries ~ CPUs > 5 PB storage > concurrent jobs per day > 60 Virtual Organisations EUIndiaGrid EUMedGrid SEE-GRID EELA BalticGrid EUChinaGrid OSG NAREGI

Barcelona, 28 March EGI Key Services Based on experience gathered during EGEE, the following key services have been found necessary for a central organisation in coordination with the NGIs –The information on the following slides are taken from existing EGEE structures and procedures as a means of explaining the concepts Operation of Infrastructure  Runs Operational Coordination Centre linking a Regional Operations Centre (Point of Presence) in each NGI  Coordinates the managed upgrades of services deployed by NGIs  Coordinates the grid security (incident response etc.)  Coordinates the grid resource accounting  Negotiates resources for user communities  General management of the User Support process  Interaction point with GEANT for grid issues  Provides documentation for end users and resource providers

Barcelona, 28 March EGI Key Services (cont) Middleware testing and certification  Integrates middleware from other sources to produce distributions  Provides first-level middleware support team  Operates beta-test services for upcoming distributions Application support  No direct support but rather coordination of NGI support groups and management of overall application lifecycle (virtuous cycle) Dissemination and outreach  Branding, media relations, production of promotional material, websites etc  Event organisation  Public outreach & surveys - Ensuring higher level of media coverage as technology matures and becomes available to more end users  Representation of NGIs in international bodies and standards groups at a policy level Training  Aid in the formation of NGIs and their management/operations  Training material repository

Barcelona, 28 March EGI/NGIs and industry Business model for how industry can commercially exploit the research infrastructure managed by EGI/NGIs is unclear –Do not have mechanisms by which clients could pay for use of resources –Use of GEANT network for commercial application is restricted Likely to see transfer of technology from research to industry by adoption/internalisation of EGI/NGI backed products and services (e.g. middleware, operations procedures/techniques) –For multi-site corporate usage or to offer a service to a set of SMEs –Several examples already exist for EGEE EGI/NGIs could subcontract infrastructure support to industry and make use of commercial software as standards evolve

Barcelona, 28 March EGI Managed Resource Centres Given the existence of such an e-Infrastructure, EGI managed resources centres can be established: –Create shared pool of resources (CPU, disk and data curation) independent of funding for specific user communities –Joint capital funding from NGIs and EU as part of new resource infrastructure –Selection based on bids against a defined service level agreement –Business models with pay-per-usage to cover operational and depreciation costs –Would create/test example business models for potential future commercial supply and/or exploitation

Barcelona, 28 March EGI Governance and Funding Governance –Organisation with its own legal identity –NGIs are the stakeholders  NGIs would form the governing council  Annual reviews by independent experts nominated by the EU What to Fund –Basic infrastructure and its operation including national Points-of- Presence, regional resource centres and central organisation How to Fund –Basic funding by NGIs (50%) and EU (50%) ? –EU willing to fund preparatory project to set-up EGI (~12 months?) –Full EU co-funding (FP7) to start in 2008/9

Barcelona, 28 March Possible parallel streams for Grids Structures 1.Sustainable e-Infrastructure –Targeted non-competitive call open to EGI & NGIs only –Linked to established roadmap as proposed by ESFRI –Long-term cycle (up to 7 years) –Production/operations focus –Application independent 2.On-going developments –Competitive calls focused on IST priorities –Examples: projects such as EGEE, OMII-Europe, NextGrid, DILIGENT etc. –Limited duration Complementary streams where successful developments (2) should become part of the sustainable infrastructure (1) Possible combination of use of EU instruments for on-going and new infrastructures

Barcelona, 28 March Summary The need for a European e-Infrastructure has been identified The current structures are reaching their limits A model committing the National Grid Initiatives and building a central organisation is proposed – your input and feedback is actively sought Such a scheme will ensure a sustainable e- Infrastructure for research and help maintain Europe’s leading position

Barcelona, 28 March Details on EGI key services The following slides provide greater detail of the key services proposed for the central organisation in coordination with the NGIs These details are based on the experience gathered through EGEE and related projects

Barcelona, 28 March JSPG: Joint Security Policy Group OSCT: Operational Security Coordination Team Operations Coordination Regional Operations Centre … … Resource Centre Resource Centre … Resource Centre Resource Centre … Operations Coordination Centre OSCT Coordination, Middleware deployment Operational security coordination JSPG Coordination, Middleware deployment Coordination, Middleware deployment EGI NGI local

Barcelona, 28 March Regional Operations Centre (ROC) Distributed responsibility for operations: One ROC in each NGI –A grid “Point of presence” –Manage daily grid operations – oversight, troubleshooting  “Operator on Duty” with weekly rota between NGIs –Run infrastructure services (not applications themselves) –Support for user and operations issues –Provide regional knowledge and adaptations Operations coordination –Regular operations & managers meetings –Series of Operations Workshops Procedures described in Operations Manual –Introducing new sites –Site downtime scheduling –Suspending a site –Escalation procedures –Etc.

Barcelona, 28 March ROC and Site work to resolve the problem Regional Operations Centre …… Resource Centre Resource Centre … OSCT Grid Operator on-duty Monitoring shows a problem Operator submits a GGUS ticket against the ROC and cc’s the site. The ticket is followed until it is solved Operations support workflows Regional Operations Centre Resource Centre Resource Centre … EGI NGI Local support

Barcelona, 28 March Coordination of Grid Security Joint Security Policy Group –Across many Grid infrastructures –Policy Set: Policy Revisions Grid Acceptable Use Policy (AUP) –for all VO members using many Grid infrastructures  EGEE, OSG, SEE-GRID, DEISA, national Grids… VO Security –responsibilities for VO managers and members –VO AUP to tie members to Grid AUP accepted at registration Incident Handling and Response –defines basic communications paths –defines requirements  reporting  response  protection of data  analysis –not to replace or interfere with local response plans Security & Availability Policy Usage Rules Certification Authorities Audit Requirements Incident Response User Registration & VO Management Application Development & Network Admin Guide VO Security

Barcelona, 28 March Operational Security Coordination Team (OSCT) What it is not: –Not focused on middleware security architecture –Not focused on vulnerabilities (see Vulnerabilities Group) Focus on Incident Response Coordination –Assume it’s broken, how do we respond? –Planning and Tracking Focus on ‘Best Practice’ –Advice –Monitoring –Analysis Coordinators for each ROC SSC1 - Job Trace SSC2 - Storage Audit Infrastructure HANDBOOK Incident Response Policy Procedures Resources Reference Playbook Security Service Challenge Infrastructure Agents Deployment Monitoring Tools 3 strategies OSCT membership  ROC security contacts

Barcelona, 28 March Vulnerability Group Purpose: inform developers, operations, site managers of vulnerabilities as they are identified and encourage them to produce fixes or to reduce their impact Set up (private!) database of vulnerabilities –To inform sites and developers Urgent action  OSCT to manage After reaction time (45 days) –Vulnerability and risk analysis given to OSCT to define action – publication? –Will not publish vulnerabilities with no solution Report progress and statistics on vulnerabilities Balance between open responsible public disclosure and creating security issues with precipitous publication

Barcelona, 28 March General management of the User Support process A portal with a well structured information and updated documentation Knowledgeable experts Correct, complete and responsive support Tools to help resolve problems –search engines –monitoring applications –resources status Examples, templates, specific distributions for software of interest Interface with other Grid support systems Connection with developers, deployment, operation teams Assistance during production use of the grid infrastructure

Barcelona, 28 March Central Application (GGUS) Deployment Support Middleware Support Network Support Operations Support TPM ROC 1 ROC 10 ROC… VO Support Interface Webportal The Support Model “Regional Support with Central Coordination" The ROCs, VOs are connected via a central integration platform provided Regional Support units User Support units Technical Support units

Barcelona, 28 March Negotiates resources for user communities Brings together VOs and NGIs Negotiate for services and resources  Run services on behalf of the VO  Provide compute and/or storage resources NOTE: Computational and storage resources are not funded by EGI

Barcelona, 28 March Interaction point with GEANT for grid issues Technical Network Liaison Committee to address grid issues with GEANT/NRENs Definition and establishment of Service Level Agreements for end-to-end services Joint operation of ENOC (e-Infrastructure Network Operations Centre) Deployment of network performance mgmt tools Coordination of input to GGF Network Measurements Working Group

Barcelona, 28 March Integration VDT/OSG OMII- Europe EGEE Etc. … Testing & Certification Support, analysis, debugging Production service deployment Pre-production service Middleware providers Integration/testing Certification activities Middleware testing and certification feedback