Introduction to Grid Computing René Kobler Institute of Graphics and Parallel Processing Johannes Kepler Univ. Linz.

Slides:



Advertisements
Similar presentations
High Performance Computing Course Notes Grid Computing.
Advertisements

EInfrastructures (Internet and Grids) US Resource Centers Perspective: implementation and execution challenges Alan Blatecky Executive Director SDSC.
IDC HPC User Forum Conference Appro Product Update Anthony Kenisky, VP of Sales.
Introduction to Grids and Grid applications Gergely Sipos MTA SZTAKI
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
ICT and Civil ProtectionSenigallia, June 2007 A Service-Oriented Middleware for EU Civil Protection cooperation Regione Marche.
CoreGRID Workpackage 5 Virtual Institute on Grid Information and Monitoring Services Authorizing Grid Resource Access and Consumption Erik Elmroth, Michał.
Parallel Programming Henri Bal Rob van Nieuwpoort Vrije Universiteit Amsterdam Faculty of Sciences.
Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam.
Introduction and Overview “the grid” – a proposed distributed computing infrastructure for advanced science and engineering. Purpose: grid concept is motivated.
Computer Science Department 1 Load Balancing and Grid Computing David Finkel Computer Science Department Worcester Polytechnic Institute.
The LHC Computing Grid – February 2008 The Worldwide LHC Computing Grid Dr Ian Bird LCG Project Leader 15 th April 2009 Visit of Spanish Royal Academy.
What is Grid Computing? Grid Computing is applying the resources of many computers in a network to a single entity at the same time;  Usually to a scientific.
Milos Kobliha Alejandro Cimadevilla Luis de Alba Parallel Computing Seminar GROUP 12.
Globus Ian Foster and Carl Kesselman Argonne National Laboratory and University of Southern California
Knowledge Environments for Science: Representative Projects Ian Foster Argonne National Laboratory University of Chicago
Parallel Programming Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences.
Grid Computing Net 535.
A.V. Bogdanov Private cloud vs personal supercomputer.
US NITRD LSN-MAGIC Coordinating Team – Organization and Goals Richard Carlson NGNS Program Manager, Research Division, Office of Advanced Scientific Computing.
SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid.
A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster
DISTRIBUTED COMPUTING
Unit – I CLIENT / SERVER ARCHITECTURE. Unit Structure  Evolution of Client/Server Architecture  Client/Server Model  Characteristics of Client/Server.
Jarek Nabrzyski, Ariel Oleksiak Comparison of Grid Middleware in European Grid Projects Jarek Nabrzyski, Ariel Oleksiak Poznań Supercomputing and Networking.
The Globus Project: A Status Report Ian Foster Carl Kesselman
The Anatomy of the Grid Mahdi Hamzeh Fall 2005 Class Presentation for the Parallel Processing Course. All figures and data are copyrights of their respective.
Virtual Data Grid Architecture Ewa Deelman, Ian Foster, Carl Kesselman, Miron Livny.
1 4/23/2007 Introduction to Grid computing Sunil Avutu Graduate Student Dept.of Computer Science.
DataTAG Research and Technological Development for a Transatlantic Grid Abstract Several major international Grid development projects are underway at.
Service - Oriented Middleware for Distributed Data Mining on the Grid ,劉妘鑏 Antonio C., Domenico T., and Paolo T. Journal of Parallel and Distributed.
Perspectives on Grid Technology Ian Foster Argonne National Laboratory The University of Chicago.
Introduction to Grid Computing Ed Seidel Max Planck Institute for Gravitational Physics
Grid Middleware Tutorial / Grid Technologies IntroSlide 1 /14 Grid Technologies Intro Ivan Degtyarenko ivan.degtyarenko dog csc dot fi CSC – The Finnish.
Grid Computing & Semantic Web. Grid Computing Proposed with the idea of electric power grid; Aims at integrating large-scale (global scale) computing.
GRID ARCHITECTURE Chintan O.Patel. CS 551 Fall 2002 Workshop 1 Software Architectures 2 What is Grid ? "...a flexible, secure, coordinated resource- sharing.
Authors: Ronnie Julio Cole David
Key prototype applications Grid Computing Grid computing is increasingly perceived as the main enabling technology for facilitating multi-institutional.
The Globus Toolkit®: The Open Source Solution for Grid Computing
…building the next IT revolution From Web to Grid…
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
Grid User Interface for ATLAS & LHCb A more recent UK mini production used input data stored on RAL’s tape server, the requirements in JDL and the IC Resource.
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
Introduction to Grids By: Fetahi Z. Wuhib [CSD2004-Team19]
Enabling e-Research in Combustion Research Community T.V Pham 1, P.M. Dew 1, L.M.S. Lau 1 and M.J. Pilling 2 1 School of Computing 2 School of Chemistry.
INFSO-RI Enabling Grids for E-sciencE EGEE is a project funded by the European Union under contract INFSO-RI Grid Accounting.
1 Observations on Architecture, Protocols, Services, APIs, SDKs, and the Role of the Grid Forum Ian Foster Carl Kesselman Steven Tuecke.
7. Grid Computing Systems and Resource Management
Globus and PlanetLab Resource Management Solutions Compared M. Ripeanu, M. Bowman, J. Chase, I. Foster, M. Milenkovic Presented by Dionysis Logothetis.
Outline Why this subject? What is High Performance Computing?
Securing the Grid & other Middleware Challenges Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer.
GRID ANATOMY Advanced Computing Concepts – Dr. Emmanuel Pilli.
3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 2.
GRIDSTART a European GRID coordination attempt Fabrizio Gagliardi CERN.
The Globus Toolkit The Globus project was started by Ian Foster and Carl Kesselman from Argonne National Labs and USC respectively. The Globus toolkit.
INTRODUCTION TO GRID & CLOUD COMPUTING U. Jhashuva 1 Asst. Professor Dept. of CSE.
10-Feb-00 CERN HepCCC Grid Initiative ATLAS meeting – 16 February 2000 Les Robertson CERN/IT.
Clouds , Grids and Clusters
SuperComputing 2003 “The Great Academia / Industry Grid Debate” ?
Grid Computing.
University of Technology
GRID COMPUTING PRESENTED BY : Richa Chaudhary.
Grid Computing B.Ramamurthy 9/22/2018 B.Ramamurthy.
ExaO: Software Defined Data Distribution for Exascale Sciences
EGI Webinar - Introduction -
The Anatomy and The Physiology of the Grid
The Anatomy and The Physiology of the Grid
Defining the Grid Fabrizio Gagliardi EMEA Director Technical Computing
Vrije Universiteit Amsterdam
Presentation transcript:

Introduction to Grid Computing René Kobler Institute of Graphics and Parallel Processing Johannes Kepler Univ. Linz

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 2 Outline  History, Motivation  Basic concepts  The EU Data Grid Project  The Austrian Grid

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 3 Why High Performance Computing?  Mathematical models get more and more complex  Before industry is able to construct products simulations are required  More complex models → Simulation time increases  Solution: Parallelize your problems

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 4 Example: Grand Challenges  Most complex computing problems nowadays (broad scientific consensus)  Examples:  n-body simulations in astrophysics  Protein folding (basics for understanding the fundamentals of life)  Meteorological simulations (weather forecast)

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 5 How to program a parallel system  Analyse your problem size  Distribute problem on different processes / threads  Minimize costly communication!  Techniques:  Shared memory  Message passing  “mixed-mode programming”

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 6 Shared Memory vs. Message Passing  Shared Memory Pro's:  Shared address space is easy to program  Message Passing programs requires often tremendous restructuring of code  Message Passing Pro's:  Efficiency! Message Passing Codes are usually much faster than equivalent shared-memory codes

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 7 Comparing HPC Systems   Since 1993  Update every June and November, respectively  Next List at SC Seattle, Nov. 12 – 18  Every system executes Linpack benchmark  Solving a dense system of linear equations  Important ratios:  R max = maximal Linpack performance achieved  R peak = theoretical peak performance

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 8 Current TOP 500 ( )  IBM manufactures 6 of the top 10 systems  Trend:  1993 → many vector- processors  Today → commodity processors (Intel, PowerPC, AMD)

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development. 9 Mare Nostrum  Fastest European Supercomputer  Housed in a majestic chapel - 2,282 IBM eServer BladeCenter JS20 blade server - housed in 163 BladeCenter chassis - 4, bit IBM Power PC 970FX processors Source: IBM

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Problems with HPC Systems  Large-Scale HPC systems are traditionally very expensive even with the usage of commodity HW  Lower sophisticated countries do not achieve access to such systems  Solution: Bundle distributed resources for common usage!  Problem: We need higher network bandwith  Moore's Law vs. Gilder's Law

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Moore's Law vs. Gilder's Law  Moore's Law:  Sloppy formulated: Processor power doubles every 18 months (orginally formulated in 1965, updated in 1975)  Gilder's Law:  Bandwith of communication systems triples every 12 months  We cannot focus only on processing power!  Gilder's Law even affects the internet's bandwith  Future on Distributed Computing

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Next step: Distributed Computing  Often realized in form of cluster computing  Using the internet as communication media:  Disadvantage: lack of intransparency  Therefore: Grid Computing

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Grid Computing: Idea  The electric power grid served as archetype for the term “grid computing”  The electric power grid is used by simply plugging in electrical devices.  The computational grid should be used by simply submitting our problem  Idea in the mid-1990s by Ian Foster (UIC) and Carl Kesselman (USC)

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Grid concept  “Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organisations” (Foster I., Kesselman C., Tuecke S., “The Anatomy of the Grid”, Int. Journal of Supercomputer Applications, 15(3), 2001)  Sharing → not only file exchange, but rather direct access to computers, software, data, and other resources (e.g. Sensors,...)  Sharing rules  Virtual Organization (VO) → A set of individuals and/or institutions defined by sharing rules

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development What do we need?  Protocols, services, tools to address challenges that arise when building scalable VOs.  Security solutions  Management of credentials and policies  Resource management protocols  Information query protocols  Services that provide configuration and status information about resources, organizations and services  Data management services

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Virtual organizations vs. actual organizations Each resource owner makes resources available, subject to constraints on when, where, and what can be done

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Constraints  Requires mechanisms for expressing policies  for establishing the identity of a consumer or resource → Authentication  for determining whether an operation is consistent with applicable sharing relationships → Authorization

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Grid Architecture  Requires to be able to establish sharing relation- ships among any potential participants  Interoperability is thus the central issue → common protocols  Grid Architecture = protocol architecture defining basic mechanisms by which VO users and resources negotiate, establish, manage, and exploit sharing relationships

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Grid Protocol Architecture

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Layers of the Grid protocol architecture  Connectivity: core communication and authentication protocols required for grid-specific network transactions.  Resource: secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources.  Collective: protocols and services of global nature to capture interactions across collections of resources.

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Case Study: EU Data Grid Project  Exploit and build the next generation computing infrastructure providing intensive computation and analysis of shared large-scale databases  Enable data intensive sciences by providing world wide Grid test beds to large distributed scientific organizations.  Start: Jan 1 st 2001, End: Dec. 31 st 2003  Applications/Communities: HEPHY, Earth Obs., Biology

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Specific Project Objectives  Middleware for fabric & grid management  Large scale testbed  Production quality demonstrations  Contribute to Open Standards and International Bodies (GGF, Industry & Research Forum)

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Next Steps -> LHC Grid (starts 2007)  Data Grid was successful -> showed, that Grids can cope with large amounts of data  Next Step: Large Hadron Collider Grid  LHC: Largest scientific instrument on the planet (located at CERN) -> produces 15 TB/year  4-Tier architecture  Tier-0 is located at CERN and collects all data  Distribute after initial proc. to Tier-1 centres with large storage capabilities

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development ,6 km

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development The Austrian Grid  Main Target: Pioneering Grid Computing in Austria  Main Idea: Demonstration of Usefulness  Main Tasks:  Building a prototype Grid infrastructure in Austria  Improving existing Grid software by high-level extensions  Development and usage of Grid applications  Inviting potential users to use Grid technology and supporting them  Representing a contact institution for future partners

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Middleware Extensions (1)  First figure out the requirements of the applications  Determine the extensions!  Modules closely related to applications must be implemented by application programmers  General parts must be implemented by computer scientists

Austrian Grid is a project funded by the bm:bwk (Federal Ministry for Education, Science and Culture) after recommendation by the Austrian Council for Research and Technology Development Middleware Extensions (2)