INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org What is Grid Computing? Mike Mineter Training Outreach and Education National e-Science Centre.

Slides:



Advertisements
Similar presentations
The National Grid Service and OGSA-DAI Mike Mineter
Advertisements

Current status of grids: the need for standards Mike Mineter TOE-NeSC, Edinburgh.
INFSO-RI Enabling Grids for E-sciencE Overview of e-Infrastructure Mike Mineter Training Outreach and Education National e-Science.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Tutorial How to get started.
INFSO-RI Enabling Grids for E-sciencE Concepts of grid computing Guy Warner NeSC Training Team
INFSO-RI Enabling Grids for E-sciencE JRA3 2 nd EU Review Input David Groep NIKHEF.
CERN Krakow 2001 F. Gagliardi - CERN/IT 1 RTD efforts in Europe by Kyriakos Baxevanidis Foster cohesion, interoperability, cross- fertilization of knowledge,
EGEE-II INFSO-RI Enabling Grids for E-sciencE An Introduction to Grid Computing Presented by…. With thanks to EGEE colleagues for.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Tutorial Welcome!!
INFSO-RI Enabling Grids for E-sciencE What is Grid Computing? Mike Mineter Training Outreach and Education National e-Science Centre.
SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Overview of the EGEE project and the gLite middleware Gergely Sipos MTA SZTAKI
Open Source Grid Computing in the Finance Industry Alex Efimov STFC Kite Club Knowledge Exchange Advisor UK CERN Technology Transfer Officer
CI Days: Planning Your Campus Cyberinfrastructure Strategy Russ Hobby, Internet2 Internet2 Member Meeting 9 October 2007.
CS e-Science & Grid Computing - introduction - What is e-Science? What is the Grid? Grid middleware.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Configuring and Maintaining EGEE Production.
DAME: Distributed Engine Health Monitoring on the Grid
EGEE-II INFSO-RI Enabling Grids for E-sciencE An Overview of Grid Computing Richard Hopkins Training Outreach and Education National.
From GEANT to Grid empowered Research Infrastructures ANTONELLA KARLSON DG INFSO Research Infrastructures Grids Information Day 25 March 2003 From GEANT.
DAME: The route to commercialisation Tom Jackson University of York.
Jarek Nabrzyski, Ariel Oleksiak Comparison of Grid Middleware in European Grid Projects Jarek Nabrzyski, Ariel Oleksiak Poznań Supercomputing and Networking.
What is Cyberinfrastructure? Russ Hobby, Internet2 Clemson University CI Days 20 May 2008.
Grid Technologies  Slide text. What is Grid?  The World Wide Web provides seamless access to information that is stored in many millions of different.
INFSO-RI Enabling Grids for E-sciencE What is Grid Computing? Richard Hopkins NGS Induction – St Andrews University,
Training and the NGS Mike Mineter
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE – paving the way for a sustainable infrastructure.
Interoperability Grids, Clouds and Collaboratories Ruth Pordes Executive Director Open Science Grid, Fermilab.
Security, Authorisation and Authentication.
DAME: A Distributed Diagnostics Environment for Maintenance Duncan Russell University of Leeds.
Data and storage services on the NGS Mike Mineter Training Outreach and Education
EGEE-II INFSO-RI Enabling Grids for E-sciencE An Introduction to the EGEE Project Presented by Min Tsai ISGC 2007, Taipei With thanks.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Next steps with EGEE EGEE training community.
Grid Middleware Tutorial / Grid Technologies IntroSlide 1 /14 Grid Technologies Intro Ivan Degtyarenko ivan.degtyarenko dog csc dot fi CSC – The Finnish.
SEEK Welcome Malcolm Atkinson Director 12 th May 2004.
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
Authors: Ronnie Julio Cole David
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE II: an eInfrastructure for Europe and.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Application Development on Grids Gergely Sipos MTA SZTAKI Hungarian.
EGEE-II INFSO-RI Enabling Grids for E-sciencE The GILDA training infrastructure.
Cyberinfrastructure What is it? Russ Hobby Internet2 Joint Techs, 18 July 2007.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
Campus grids: e-Infrastructure within a University Mike Mineter National e-Science Centre 14 February 2006.
The National Grid Service Mike Mineter.
Presentation of the A particle collision = an event Physicist's goal is to count, trace and characterize all the particles produced and fully.
EGEE is a project funded by the European Union under contract IST Grid computing Assaf Gottlieb Tel-Aviv University assafgot tau.ac.il
INFSO-RI Enabling Grids for E-sciencE What is Grid Computing? Richard Hopkins Training Outreach and Education National e-Science.
INFSO-RI Enabling Grids for E-sciencE GILDA and GENIUS Guy Warner NeSC Training Team An induction to EGEE for GOSC and the NGS NeSC,
Enabling, facilitating and delivering quality training in the UK and Internationally Introduction to e-science concepts Mike Mineter Training Outreach.
Cyberinfrastructure Overview Russ Hobby, Internet2 ECSU CI Days 4 January 2008.
INFSO-RI Enabling Grids for E-sciencE Overview of e-Infrastructure Mike Mineter Training Team National e-Science Centre
Toward a common data and command representation for quantum chemistry Malcolm Atkinson Director 5 th April 2004.
EGEE-II INFSO-RI Enabling Grids for E-sciencE What is Grid Computing? Mike Mineter Training Outreach and Education National e-Science.
The National Grid Service Mike Mineter.
INFSO-RI Enabling Grids for E-sciencE Middleware Concepts and Services in Campus, National and International Grids Mike Mineter.
NERC e-Science Meeting Malcolm Atkinson Director & e-Science Envoy UK National e-Science Centre & e-Science Institute 26 th April 2006.
GRIDSTART a European GRID coordination attempt Fabrizio Gagliardi CERN.
GRIDSTART Brussels 20/9/02 1www.gridstart.org GRIDSTART and European activities Dr Francis Wray EPCC The University of Edinburgh.
EGEE-II INFSO-RI Enabling Grids for E-sciencE A Glance Towards the Future Mike Mineter Training Outreach and Education University.
All Hands Meeting 2005 BIRN-CC: Building, Maintaining and Maturing a National Information Infrastructure to Enable and Advance Biomedical Research.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Overview of gLite, the EGEE middleware Mike Mineter Training Outreach Education National.
Campus grids: e-Infrastructure within a University Mike Mineter National e-Science Centre 22 February 2006.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Authentication, Authorisation and Security Emidio Giorgio INFN Catania.
INFSO-RI Enabling Grids for E-sciencE Sofia, 17 March 2009 Security, Authentication and Authorisation Mike Mineter Training, Outreach.
Bob Jones EGEE Technical Director
Long-term Grid Sustainability
Tour of CERN Computer Center
Tour of CERN Computer Center
Overview of the EGEE project and middleware
Presentation transcript:

INFSO-RI Enabling Grids for E-sciencE What is Grid Computing? Mike Mineter Training Outreach and Education National e-Science Centre

2 You are welcome to re-use these slides. We ask only that you let us know, by to

Enabling Grids for E-sciencE INFSO-RI Contents Introduction to –e-Research and e-Science –Grids –e-Infrastructure Grid concepts Grids - Where are we now?

Enabling Grids for E-sciencE INFSO-RI ‘e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ John Taylor Director General of Research Councils Office of Science and Technology

Enabling Grids for E-sciencE INFSO-RI ‘e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ Networks + Grids –Networks connect resources –Grids enable “virtual computing”

6 IRAS 25  2MASS 2  DSS Optical IRAS 100  NVSS 20cm GB 6cm ROSAT ~keV WENSS 92cm Observations made across entire electromagnetic spectrum  e.g. different views of a local galaxy Need all of them to understand physics fully Databases are located throughout the world Peter Clarke Virtual Observatories

Biomedical Research Informatics Delivered by Grid Enabled Services Synteny Grid Service blast + VO Authorisation Information Integrator

Enabling Grids for E-sciencE INFSO-RI Engine flight data Airline office Maintenance Centre European data center London Airport New York Airport American data center Grid Diagnostics Centre “A Significant factor in the success of the Rolls-Royce campaign to power the Boeing 7E7 with the Trent 1000 was the emphasis on the new aftermarket support service for the engines provided via DS&S. Boeing personnel were shown DAME as an example of the new ways of gathering and processing the large amounts of data that could be retrieved from an advanced aircraft such as the 7E7, and they were very impressed”, DS&S 2004 XTO Engine Model Case Based Reasoning Signal Data Explorer Companies: Rolls-Royce DS&S Cybula DAME: Grid based tools and Infer- structure for Aero-Engine Diagnosis and Prognosis Universities: York, Leeds, Sheffield, Oxford Follow-on project: BROADEN

climateprediction.net and GENIE Largest climate model ensemble >45,000 users, >1,000,000 model years 10K 2K Response of Atlantic circulation to freshwater forcing

UK Grid for Particle Physics GridPP detectors, 2/3/06

Enabling Grids for E-sciencE INFSO-RI Connecting people: Access Grid Microphones Cameras

Enabling Grids for E-sciencE INFSO-RI What is e-Research? Collaborative research that is made possible by the sharing across the Internet of resources (data, instruments, computation, people’s expertise...) –Crosses organisational boundaries –Often very compute intensive –Often very data intensive –Sometimes large-scale collaboration Began with focus in the “big sciences” hence initiatives are often badged as “e-science” Relevance of “e-science technologies” to new user communities (social science, arts, humanities…) led to the term “e-research”

Enabling Grids for E-sciencE INFSO-RI e-Research and Grids People with shared goals Improvised cooperation File exchange ssh access to run programs Enabled by networks: national, regional and International: GEANT Collaborative “virtual computing” Sharing data, computers, software Enabled by Grids: National, regional International: EGEE grid

Grids: a foundation for e-Research sensor nets Shared data archives computers software colleagues instruments Grid enabling a whole-system approach Diagram derived from Ian Foster’s slide

Enabling Grids for E-sciencE INFSO-RI What is Grid Computing? The grid vision is of “Virtual computing” (+ information services to locate computation, storage resources) –Compare: The web: “virtual documents” (+ search engine to locate them) MOTIVATION: collaboration through sharing resources (and expertise) to expand horizons of –Research –Commerce – engineering, … –Public service – health, environment,…

Enabling Grids for E-sciencE INFSO-RI The Grid Metaphor

Enabling Grids for E-sciencE INFSO-RI What is e-Infrastructure? – Political view A shared resource –That enables science, research, engineering, medicine, industry, … –It will improve UK / European / … productivity  Lisbon Accord 2000  E-Science Vision SR2000 – John Taylor –Commitment by UK government  Sections –Always there  c.f. telephones, transport, power, internet

Enabling Grids for E-sciencE INFSO-RI What is e-Infrastructure? Grids: permit resource sharing across administrative domains Networks: permit communication across geographical distance Supporting organisations –Operations for grids, networks Resources –Computers –Digital libraries –Research data –Instruments Middleware –Authentication, Authorisation –Registries, search engines –Toolkits, environments  E.g. for collaboration Network infrastructure & Resources Operations, Support and training Collaboration Grid

Enabling Grids for E-sciencE INFSO-RI Global Drivers of e-Research Digital technology – exponential growth - e.g. bandwidth Opportunities for e-Infrastructure to support faster, better, different research –Sharing expertise  Support for cooperation and communication –Sharing computation services  E.g. to serve occasional peaks of high demand for computation (especially trivially parallelisable ones) –Sharing data  New sensors and instruments  Databases Based on an infrastructure that requires and enables multidisciplinary research  Requires: IT + domain specialists  Enables: New interdisciplinary research

Enabling Grids for E-sciencE INFSO-RI What is Grid computing? The term “Grid” has become popular! –Sometimes in Industry : “Grids” = clusters  Motivations: better use of resources; scope for commercial services –Also used to refer to the harvesting of donated, unused compute cycles  Climateprediction.net) –These are e-Infrastructure but are not “grids” from the e- Research viewpoint!

Enabling Grids for E-sciencE INFSO-RI Grid concepts

Enabling Grids for E-sciencE INFSO-RI Virtual organisations and grids What’s a Virtual Organisation? –People in different organisations seeking to cooperate and share resources across their organisational boundaries E.g. A research collaboration Each grid is an infrastructure enabling one or more “virtual organisations” to share and access resources Key concept: The ability to negotiate resource-sharing arrangements among a set of participating parties (providers and consumers) and then to use the resulting resource pool for some purpose. (Ian Foster)

Enabling Grids for E-sciencE INFSO-RI INTERNET Virtual organisations negotiate with sites to agree access to resources Grid middleware runs on each shared resource to provide –Data services –Computation services –Single sign-on Distributed services (both people and middleware) enable the grid Typical current grid

Enabling Grids for E-sciencE INFSO-RI Grid Middleware When using a PC or workstation you –Login with a username and password (“Authentication”) –Use rights given to you (“Authorisation”) –Run jobs –Manage files: create them, read/write, list directories Components are linked by a bus Operating system One admin. domain When using a Grid you –Login with digital credentials – single sign- on (“Authentication”) –Use rights given you (“Authorisation”) –Run jobs –Manage files: create them, read/write, list directories Services are linked by the Internet Middleware Many admin. domains

Enabling Grids for E-sciencE INFSO-RI Empowering VO’s Where computer science meets the application communities! VO-specific developments: –Portals –Virtual Research Environments –Semantics, ontologies –Workflow –Registries of VO services Basic Grid services: AA, job submission, info, … Higher-level grid services (brokering,…) Application toolkits, … Application Production grids provide these services.

Enabling Grids for E-sciencE INFSO-RI Example – Biomedical applications Biomedical community and the Grid, EGEE User Forum, March 1 st 2006, I. Magnin

Enabling Grids for E-sciencE INFSO-RI Workflow example Taverna in MyGrid “allows the e-Scientist to describe and enact their experimental processes in a structured, repeatable and verifiable way” GUI Workflow language enactment engine

Enabling Grids for E-sciencE INFSO-RI The many scales of grids Campus grids Regional grids (e.g. Brunel/Westminster Grid Alliance) National grids (e.g. National Grid Service) International grid (EGEE) Wider collaboration greater resources National datacentres, HPC, instruments Institutes’ data; Condor pools, clusters International instruments,.. Desktop

Enabling Grids for E-sciencE INFSO-RI Access service Access serviceHow users logon to a Grid Computing Element (CE) Computing Element (CE): A batch queue on a site’s computers where the user’s job is executed Storage Element (SE) Storage Element (SE): provides (large-scale) storage for files Resource Broker (RB) Resource Broker (RB): Service that matches the user’s requirements with the available resources on a Grid Main components Information System Information System: Characteristics and status of resources

Enabling Grids for E-sciencE INFSO-RI Who provides the resources?! ServiceProviderNote Access service User / institute/ VO / grid operations Computer with client software Resource Broker (RB) VO / grid operations(No NGS-wide RB exists) Information System Information System:Grid operations Computing Element (CE) VO / sometimes centralised provision also Scalability requires that VOs provide resources to match average need Storage Element (SE) ditto “VO”: virtual organisation “Grid operations”: funded effort

Enabling Grids for E-sciencE INFSO-RI Current EGEE grid File Replica Catalogue Logging & Book-keeping ResourceBroker StorageResource ComputingResource = batch queue InformationService Job Status Datasets info Author. &Authen. Job Submit Event Job Query Job Status Input files Input Output Output files Publish Resource info User/Grid interface

Enabling Grids for E-sciencE INFSO-RI EGEE is … EU-funded project that has established the largest multi-VO production grid in the world! What’s happening now? What resources are connected?

Enabling Grids for E-sciencE INFSO-RI Grid security and trust -1 Providers of resources (computers, databases,..) need risks to be controlled: they are asked to trust users they do not know –They trust a VO –The VO trusts its members User’s need –single sign-on: to be able to logon to a machine that can pass the user’s identity to other resources –To trust owners of the resources they are using Build middleware on layer providing: –Authentication: know who wants to use resource –Authorisation: know what the user is allowed to do –Security: reduce vulnerability, e.g. from outside the firewall –Non-repudiation: knowing who did what The “Grid Security Infrastructure” middleware is the basis of (most) production grids

Enabling Grids for E-sciencE INFSO-RI Grid security and trust -2 Achieved by Certification: –User’s identity has to be certified by one of the national Certification Authorities (CAs) User –User joins a VO –Digital certificate is basis of AA –Identity passed to resources you use, where it is mapped to a local account Policies express the rights for a Virtual Organization to use resources

Enabling Grids for E-sciencE INFSO-RI The Role of the Virtual Organisation (VO) Compute Center VO Service slide based on presentation given by Carl Kesselman at GGF Summer School 2004

Enabling Grids for E-sciencE INFSO-RI … then where are we now? If “The Grid” vision leads us here…

Enabling Grids for E-sciencE INFSO-RI Grid projects - ~ 2003 Many Grid development efforts — all over the world NASA Information Power Grid DOE Science Grid NSF National Virtual Observatory NSF GriPhyN DOE Particle Physics Data Grid NSF TeraGrid DOE ASCI Grid DOE Earth Systems Grid DARPA CoABS Grid NEESGrid DOH BIRN NSF iVDGL DataGrid (CERN,...) EuroGrid (Unicore) DataTag (CERN,…) Astrophysical Virtual Observatory GRIP (Globus/Unicore) GRIA (Industrial applications) GridLab (Cactus Toolkit) CrossGrid (Infrastructure Components) EGSO (Solar Physics) UK – OGSA-DAI, RealityGrid, GeoDise, Comb-e-Chem, DiscoveryNet, DAME, AstroGrid, GridPP, MyGrid, GOLD, eDiamond, Integrative Biology, … Netherlands – VLAM, PolderGrid Germany – UNICORE, Grid proposal France – Grid funding approved Italy – INFN Grid Eire – Grid proposals Switzerland - Network/Grid proposal Hungary – DemoGrid, Grid proposal Norway, Sweden - NorduGrid

Enabling Grids for E-sciencE INFSO-RI Grids: where are we now? Many key concepts identified and known Many grid projects have tested, and benefit from, these Major efforts now on establishing: –Production Grids for multiple VO’s  “Production” = Reliable with commitments to quality of service In Europe, EGEE In UK, National Grid Service In US, Teragrid and OSG  One stack of middleware that serves many research communities  Establishing operational procedures and organisation –Standards (a slow process) (e.g. Open Grid Forum, ) –Sustainable infrastructure  Move from research funding to GEANT-like model  European Grid Infrastructure federating National Grid Infrastructures

Enabling Grids for E-sciencE INFSO-RI Where are we now? –user’s view ResearchPilot projects Early adopters Routine production Unimagined possibilities Grids Networks Web Sciences, engineering Arts Humanities e-Soc-Sci Early production grids: International - EGEE Service-oriented, workflow, “legacy” data High throughput, new data Types of use:

Enabling Grids for E-sciencE INFSO-RI Where are we now?! Standards are emerging… some near acceptance and some being discarded –Standards bodies:  W3C  GGF  OASIS  IETFhttp:// –For a (slightly outdated) summary see standards/poster/ standards/poster/ Production grids are based on de-facto standards at present –Inevitably! –GT2 especially –But locks a grid into one middleware stack unable to benefit from the diverse developments of new services

Enabling Grids for E-sciencE INFSO-RI National grid initiatives now include… CroGrid

Enabling Grids for E-sciencE INFSO-RI What are Grids? - Summary Grids enable virtual computing across administrative domains –Resources share authorisation and authentication –Resources accessed thru abstractions Motivations: –Collaborative research, diagnostics, engineering, public service,.. –Resource utilisation and sharing Infrastructure for research Network infrastructure & Resource centres Operations, Support and training Collaboration Grid

Enabling Grids for E-sciencE INFSO-RI Further reading The Grid Cafe Grid Today International Science Grid This Week UK All Hands Meeting National e-Science Centre International School of Grid Computing Open Grid Forum EGEE Conference (next week) egee.org/egee_events/conferencehttp:// egee.org/egee_events/conference