EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks Steven Newhouse Director EGI.eu Project.

Slides:



Advertisements
Similar presentations
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse Technical Director, EGEE-III.
Advertisements

The EGI – a sustainable European grid infrastructure Michael Wilson STFC RAL.
EGI: European Grid Infrastructure Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III 1.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI - Identity Management Steven Newhouse Director, EGI.eu Federated Identity.
The LHC Computing Grid – February 2008 The Worldwide LHC Computing Grid Dr Ian Bird LCG Project Leader 15 th April 2009 Visit of Spanish Royal Academy.
SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.
EGI: A European Distributed Computing Infrastructure Steven Newhouse Interim EGI.eu Director.
EGI-Engage EGI-Engage Engaging the EGI Community towards an Open Science Commons Project Overview 9/14/2015 EGI-Engage: a project.
1 European policies for e- Infrastructures Belarus-Poland NREN cross-border link inauguration event Minsk, 9 November 2010 Jean-Luc Dorel European Commission.
What is EGI? The European Grid Infrastructure enables access to computing resources for European scientists from all fields of science, from Physics to.
EGI-InSPIRE RI EGI-InSPIRE RI European Grid Infrastructure: status and services for users 04/11/ Gergely Sipos.
EGI: SA1 Operations John Gordon EGEE09 Barcelona September 2009.
EGI-InSPIRE Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse EGEE’s plans for transition.
Overview & Status of the Middleware & EGI Core Proposals Steven Newhouse Interim EGI Director EGEE Technical Director 26/05/2016 Status of EGI & Middleware.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Vision for European DCIs Steven Newhouse Project Director, EGI-InSPIRE 15/09/2010.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks ?? Athens, May 5-6th 2009 Community Support.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE – paving the way for a sustainable infrastructure.
INFSO-RI Enabling Grids for E-sciencE Plan until the end of the project and beyond, sustainability plans Dieter Kranzlmüller Deputy.
European Grid Initiative – EGI Business and Technology Transfer for EGI EGI Design Study Project EGEE’09 Barcelona September 2009.
This document produced by Members of the Helix Nebula Partners and Consortium is licensed under a Creative Commons Attribution 3.0 Unported License. Permissions.
Enabling Grids for E-sciencE 1 EGEE III Project Prof.dr. Doina Banciu ICI Bucharest GRID activities in RoGrid Consortium.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse Technical Director CERN.
RI EGI-InSPIRE RI EGI Future activities Peter Solagna – EGI.eu.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE-EGI Grid Operations Transition Maite.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Bob Jones EGEE project director CERN.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The use of grids for research infrastructures.
European Grid Initiative Per Öster Director Application Services, CSC – IT Center for Science Ltd, Finland EGI Council Chair Chair EGI.eu Executive Board.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SA1: Grid Operations Maite Barroso (CERN)
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks David Fergusson, Emidio Giorgio, Gergely.
EGEE-III-INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE-III All Activity Meeting Brussels,
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse Technical Director CERN.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGI Operations Tiziana Ferrari EGEE User.
EGI-InSPIRE Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III 1GDB - December 2009.
EMI INFSO-RI EMI Roadmap to Standardization and DCI Collaborations Alberto Di Meglio (CERN) Project Director.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Robin McConnell NA3 Activity Manager 02.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Robin McConnell NA3 Activity Manager 28.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks User Support for Distributed Computing Infrastructures.
European Middleware Initiative (EMI) Alberto Di Meglio (CERN) Project Director.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI strategy and Grand Vision Ludek Matyska EGI Council Chair EGI InSPIRE.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE: Enabling grids for E-Science Bob Jones.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse Technical Director CERN.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Plans for PY2 Steven Newhouse Project Director, EGI.eu 30/05/2011 Future.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI UMD Roadmap Steven Newhouse 14/09/2010.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI User Support services and activities Gergely Sipos User Community Support.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI: Up and Running! Steven Newhouse Director 12/09/2010 EGI Technical Forum.
EGI-InSPIRE Project Overview1 EGI-InSPIRE Overview Activities and operations boards Tiziana Ferrari, EGI.eu Operations Unit Tiziana.Ferrari at egi.eu 1.
INFSO-RI Enabling Grids for E-sciencE EGEE general project update Fotis Karayannis EGEE South East Europe Project Management Board.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Technology Sustainability Discussion Points DCI Sustainability Meeting.
EGI-Engage EGI Webinar - Introduction - Gergely Sipos EGI.eu / MTA SZTAKI 6/26/
EGI-InSPIRE EGI-InSPIRE RI The European Grid Infrastructure Steven Newhouse Director, EGI.eu Project Director, EGI-InSPIRE 29/06/2016CoreGrid.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI A pan-European Research Infrastructure supporting the digital European Research.
EGI-InSPIRE RI EGI Compute and Data Services for Open Access in H2020 Tiziana Ferrari Technical Director, EGI.eu
EGI-InSPIRE EGI-InSPIRE RI EGI strategy towards the Open Science Commons Tiziana Ferrari EGI-InSPIRE Director at EGI.eu.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Sustainability & Business Models Sergio Andreozzi EGI.eu Policy Development.
EGI-InSPIRE RI An Introduction to European Grid Infrastructure (EGI) March An Introduction to the European Grid Infrastructure.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Role and Challenges of the Resource Centre in the EGI Ecosystem Tiziana Ferrari,
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI DCI Collaborations Steven Newhouse 15/09/2010 DCI Vision1.
Bob Jones EGEE Technical Director
Ian Bird GDB Meeting CERN 9 September 2003
Steven Newhouse EGI-InSPIRE Project Director, EGI.eu
Steven Newhouse Project Director, EGI.eu
EGEE support for HEP and other applications
Connecting the European Grid Infrastructure to Research Communities
EGI – Organisation overview and outreach
European Grid Infrastructure for Life Science
EGI Webinar - Introduction -
Steven Newhouse, EGI.eu EGI-InSPIRE Project Director
The EGI.eu Organisation
Sergio Andreozzi Strategy and Policy Manager (EGI.eu)
Presentation transcript:

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Steven Newhouse Director EGI.eu Project Director, EGI-InSPIRE Technical Director, EGEE-III European Grid Infrastructure: Enabling the Global Research Community

Enabling Grids for E-sciencE EGEE-III INFSO-RI Infrastructure (Wikipedia) Infrastructure is the basic physical and organisational structures needed for the operation of a society or enterprise, or the services and facilities necessary for an economy to function. Technical structures for us are: –Hardware: Compute, Storage, Instruments, Sensors, … –Software: Authentication, Authorisation, Accounting, … European Grid Infrastructure (BCNet) 2 The Enterprise is the research community

Enabling Grids for E-sciencE EGEE-III INFSO-RI European Grid Infrastructure European Data Grid (EDG) –Explore concepts in a testbed Enabling Grid for E-sciencE (EGEE) –Moving from prototype to production European Grid Infrastructure (EGI) –Routine usage of a sustainable e-infrastructure European Grid Infrastructure (BCNet) 3

Enabling Grids for E-sciencE EGEE-III INFSO-RI EGEE has achieved a lot! 17,000 users 139,000 LCPUs (cores) 25Pb disk 39Pb tape 12 million jobs/month +45% in a year 268 sites +5% in a year 48 countries +10% in a year 162 Virtual Organisations +29% in a year Over 20 active communities in 112 VOs European Grid Infrastructure (BCNet) 4

Enabling Grids for E-sciencE EGEE-III INFSO-RI Lessons Learned Supporting diverse communities is hard –One middleware distribution (gLite) means compromises –Focusing on a single operating model provides tensions Supporting a large operational infrastructure is costly –Communication and coordination across 260+ sites –Running hardware: compute, storage, networking,... –Running software: site, domain specific,... A production infrastructure does yield results –Recent reconstruction events from the first LHC run –In silico drug discovery searches –Fusion simulations –..... European Grid Infrastructure (BCNet) 5

Enabling Grids for E-sciencE EGEE-III INFSO-RI EGEE to EGI... what does it mean? An opportunity! –Draw a line under the experimentation in EDG & EGEE –Scope activities and structures so they are sustainable A challenge! –The technology landscape changes and we must change with it –Increasing diversity of application models and resources  Data Intensive Science is getting ever more intensive  Expand beyond core EGEE high throughput grids Integrate desktop and high performance grids  Expand technologies in response to end-user & operational needs How do virtualisation and cloud computing change things? A business model! –Add value where you can in providing a generic infrastructure –Provide an open extensible infrastructure for all European Grid Infrastructure (BCNet) 6

EGI-InSPIRE INFSO-RI What will EGI initially focus on? Continue to provide a secure reliable generic infrastructure –Integrate resources based on gLite, UNICORE, ARC, Globus,... –Leverage new technologies to provide more flexibility to users Support the user communities using the infrastructure –Assist and support the current EGEE communities –Engage with and support new structured communities e.g. ESFRI projects Improve the efficiency of the infrastructure –The number of jobs, users & data continue to increase –Utilisation and effectiveness of the resources needs to match Use new technologies to make middleware selection and operation a domain specific decision

EGI-InSPIRE INFSO-RI European Strategy Forum on Research Infrastructures Roadmap updated in 2008 Preparatory phase funding for most projects Big push in FP8 (2013 and beyond)? 44 projects covering: –Social Sciences and Humanities –Environmental Sciences –Energy –Biological and Medical Sciences –Materials and Analytical Facilities –Physical Sciences and Engineering –e-Infrastructures Data Intensive Science National commitments in European context Global collaboration and shared access Long lifetime ( years) Data Intensive Science National commitments in European context Global collaboration and shared access Long lifetime ( years)

EGI-InSPIRE INFSO-RI EGI means Innovation Deploy Technology Innovation –Distributed Computing continues to evolve Grids  Desktops  Virtualisation  Clouds  ? Enable Software Innovation –Provide reliable persistent technology platform Community tools built on the deployed technology Support Research Innovation –Infrastructure for data intensive science Support for Virtual Research Communities (e.g. ESFRI)

EGI-InSPIRE INFSO-RI Virtual Research Communities Provide infrastructure to enable European research –Use the same infrastructure for national research –Link to other infrastructures to support European researchers Unit of representation within EGI –Discipline specific –Ideally aligned with an existing external structure e.g. EIRO, community, collaboration, project,... –Could include multiple VOs –Small international research groups Support through a federated helpdesk –Continued use of GGUS –Incorporate support units from: VRCs, Operations, Technology providers, Applications,...

EGI-InSPIRE INFSO-RI EGI Collaboration NGINGI NGINGI NGINGI NGINGI Research Community Research Community Research Community Research Community EIROEIRO EIROEIRO EGI.eu Research Community NGI: National Grid Initiative EIRO: European International Research Organisation 11European Grid Infrastructure (BCNet)

EGI-InSPIRE INFSO-RI The EGI.eu Organisation Coordination for European DCI resources –Roadmap to integrate HTC, HPC, Data, Instruments,... –Policy & services needed to run a production infrastructure EGI.eu governed and owned by its stakeholders –EGI Council votes proportional to fees Fees currently set proportional to national income –Builds on resources from within its stakeholders Located in the Amsterdam Science Park –Distributed staff (~45) with a core (~50%) in Amsterdam Human coordination in Amsterdam Technical coordination with a few partners across Europe

EGI-InSPIRE INFSO-RI EGI.eu’s Services Integrated Infrastructure –Coordinates (not owns) the compute & storage resources –Resources owned by individual organisations They manage access for their user communities Deploying Innovative Technology into Production –Software for secure authorised access to resources Liaison with external (to EGI) software providers Integrated into the Unified Middleware Distribution (UMD) –EGI defined and verified interfaces Compatible software must be deployed –Interoperation within your NGI and internationally

EGI-InSPIRE INFSO-RI EGI.eu’s Services User Community Support –From a single VO to a Virtual Research Community –Provide a federated Helpdesk linking: Discipline specific support (e.g. Bio Apps) National infrastructure support (e.g. NGS) Generic services within NGIs or VRCs (e.g. Training) –Provide core services to support users Manage VOs, Application DB, Training DB –Support for Heavy User Communities Dissemination –With NGIs, VRCs, and other projects –Two Annual meetings: Users & Technology EGI Technical Forum th September 2010 in Amsterdam

EGI-InSPIRE INFSO-RI The EGI-InSPIRE Project 15 Integrated Sustainable Pan-European Infrastructure for Researchers in Europe A 4 year project with €25M EC contribution –Project cost €69M –Total Effort ~€330M –Effort: 9261 Project Partners (51) EGI.eu, 40 NGIs, 2 EIROs Asia Pacific (8 partners) Funded Un-Funded European Grid Infrastructure (BCNet)

EGI-InSPIRE INFSO-RI Be a Neutral Infrastructure Consider IP network providers –Open to any traffic from many different communities Restrictions to protect other users –Customised solutions within a generic framework Light paths on demand –Standards drive integrated deployment Hardware and fibre from many different providers And for sustainable E-Infrastructures? –Any application domain or middleware technology –A platform for domain specific innovation and use –Integration of any compliant compatible resources

EGI-InSPIRE INFSO-RI Can we learn from others? Grids have benefited from commoditisation –Hardware: HTC & HPC affordable to all –Networking: GBs can be moved over WAN –Software: Open source software comes of age How will commodity virtualisation impact us? –For transactional models  Cloud Computing: A model based on compute not data –For large distributed data-oriented models  The emergence of true ‘function shipping’?

EGI-InSPIRE INFSO-RI Evolving Service Delivery Models Move towards an interoperable cloud infrastructure –Federated pan-European infrastructure –Use standards and the established AAAA mechanisms Provide a Data-Oriented Infrastructure as a Service –Use existing high performance data storage & transfers –Empower VRCs/VOs to source and run their own services Bring new research innovations into production –Federated cloud environments (i.e. each site) –Experimenting with virtualised worker nodes in EGEE: e.g. INFN, BiG Grid, CERN, NGS, Dgrid,...

EGI-InSPIRE INFSO-RI What does this evolution mean? EGI coordinates the core infrastructure –Assessing & certifying technology for deployment –Ensure integration of the core services in Europe –Operate & manage domain specific environments If required by that domain! VOs now manage their own infrastructure –Decide what services are deployed where –Flexibility (& responsibility) to meet their own needs Deregulate and open up the infrastructure (Where it makes sense to do so!) Deregulate and open up the infrastructure (Where it makes sense to do so!)

EGI-InSPIRE INFSO-RI A long-term need for Standards Data Layer –Secure reliable data movement –Standardised access to data resources Virtualisation Layer –VMM across trust domains within agreed policies –Monitoring as important as lifecycle control Service Layer –The services that go into the virtual machine –Avoid domain specific silos & promote reuse Openness Consensus Balance Transparency Openness Consensus Balance Transparency

EGI-InSPIRE INFSO-RI Sustainability Reduce barriers for collaborative data intensive science –Integration with GEANT provides unique offering –Support to ESFRI projects and new communities Flexibility to run the services and software they need Open global collaboration of e-infrastructures providers –Domain driven collaboration with other infrastructures –Open standardised interfaces to avoid vendor lock in –Add value where we can and outsource where we can’t ‘Europe as a hub for sustainable e-science and continuous service innovation’

EGI-InSPIRE INFSO-RI Summary EGEE: –Demonstrated a production e-infrastructure EGI: –Provide a sustainable production e-infrastructure EGI.eu is now a legal entity based in Amsterdam –Supported transition for 4 years through EGI-InSPIRE Contact: EGI Technical Forum th September 2010 in Amsterdam