Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab’s GAT Grid Application Toolkit Jarek Nabrzyski et al. Poznań Supercomputing.

Slides:



Advertisements
Similar presentations
WP2: Data Management Gavin McCance University of Glasgow November 5, 2001.
Advertisements

Current status of grids: the need for standards Mike Mineter TOE-NeSC, Edinburgh.
High Performance Computing Course Notes Grid Computing.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
Cracow Grid Workshop, November 5-6, 2001 Towards the CrossGrid Architecture Marian Bubak, Marek Garbacz, Maciej Malawski, and Katarzyna Zając.
GridLab Enabling Applications on the Grid Jarek Nabrzyski et al. Poznań Supercomputing and Networking.
Network Management Overview IACT 918 July 2004 Gene Awyzio SITACS University of Wollongong.
Universität Dortmund Robotics Research Institute Information Technology Section Grid Metaschedulers An Overview and Up-to-date Solutions Christian.
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
Portals Team GridSphere and the GridLab Project Jason Novotny Michael Russell Oliver Wehrens Albert.
Cactus Code and Grid Programming Here at GGF1: Gabrielle Allen, Gerd Lanfermann, Thomas Radke, Ed Seidel Max Planck Institute for Gravitational Physics,
SC 2003 Demo, NCSA booth GridLab Project Funded by the EU (5+ M€), January 2002 – December 2004 Application and Testbed oriented Cactus Code, Triana Workflow,
Milos Kobliha Alejandro Cimadevilla Luis de Alba Parallel Computing Seminar GROUP 12.
GridLab & Cactus Joni Kivi Maarit Lintunen. GridLab  A project funded by the European Commission  The project was started in January 2002  Software.
Workload Management Massimo Sgaravatto INFN Padova.
Kelly Davis GAT: Grid Application Toolkit Kelly Davis AEI-MPG.
GridSphere for GridLab A Grid Application Server Development Framework By Michael Paul Russell Dept Computer Science University.
WP6: Grid Authorization Service Review meeting in Berlin, March 8 th 2004 Marcin Adamski Michał Chmielewski Sergiusz Fonrobert Jarek Nabrzyski Tomasz Nowocień.
SUN HPC Consortium, Heidelberg 2004 Grid(Lab) Resource Management System (GRMS) and GridLab Services Krzysztof Kurowski Poznan Supercomputing and Networking.
General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.
CGW 2003 Institute of Computer Science AGH Proposal of Adaptation of Legacy C/C++ Software to Grid Services Bartosz Baliś, Marian Bubak, Michał Węgiel,
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
GridLab A Grid Application Toolkit and Testbed IST Jarek Nabrzyski GridLab Project Coordinator Poznań.
DISTRIBUTED COMPUTING
ARGONNE  CHICAGO Ian Foster Discussion Points l Maintaining the right balance between research and development l Maintaining focus vs. accepting broader.
CoG Kit Overview Gregor von Laszewski Keith Jackson.
WP9 Resource Management Current status and plans for future Juliusz Pukacki Krzysztof Kurowski Poznan Supercomputing.
Portals & Portlets 2003, 14 July 2003 The GridLab Portal Portals & Portlets 2003 Albert Einstein Institute Jason Novotny
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Cactus Project & Collaborative Working Gabrielle Allen Max Planck Institute for Gravitational Physics, (Albert Einstein Institute)
NeSC Apps Workshop July 20 th, 2002 Customizable command line tools for Grids Ian Kelley + Gabrielle Allen Max Planck Institute for Gravitational Physics.
Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.
Applications for the Grid Here at GGF1: Gabrielle Allen, Thomas, Dramlitsch, Gerd Lanfermann, Thomas Radke, Ed Seidel Max Planck Institute for Gravitational.
Jarek Nabrzyski, Ariel Oleksiak Comparison of Grid Middleware in European Grid Projects Jarek Nabrzyski, Ariel Oleksiak Poznań Supercomputing and Networking.
GridLab: A Grid Application Toolkit and Testbed Jarosław Nabrzyski GridLab Project Manager Poznań Supercomputing and Networking Center, Poland
GridLab: A Grid Application Toolkit and Testbed
Grid Workload Management Massimo Sgaravatto INFN Padova.
Virtual Data Grid Architecture Ewa Deelman, Ian Foster, Carl Kesselman, Miron Livny.
1 4/23/2007 Introduction to Grid computing Sunil Avutu Graduate Student Dept.of Computer Science.
Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.
PROGRESS: ICCS'2003 GRID SERVICE PROVIDER: How to improve flexibility of grid user interfaces? Michał Kosiedowski.
General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.
Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”
Introduction to Grid Computing Ed Seidel Max Planck Institute for Gravitational Physics
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
Developing Applications on Today’s Grids Tom Goodale Max Planck Institute for Gravitational Physics
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
Grid Computing at PSNC Jarosław Nabrzyski Poznań Supercomputing and Networking Center (PSNC) and Information Sciences Institute, Poznan University of Technology.
GRIDS Center Middleware Overview Sandra Redman Information Technology and Systems Center and Information Technology Research Center National Space Science.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
GridLab WP-2 Cactus GAT (CGAT) Ed Seidel, AEI & LSU Co-chair, GGF Apps RG, Gridstart Apps TWG Gabrielle Allen, Robert Engel, Tom Goodale, *Thomas Radke.
CEOS Working Group on Information Systems and Services - 1 Data Services Task Team Discussions on GRID and GRIDftp Stuart Doescher, USGS WGISS-15 May 2003.
GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.
Introduction to Grids By: Fetahi Z. Wuhib [CSD2004-Team19]
Globus and PlanetLab Resource Management Solutions Compared M. Ripeanu, M. Bowman, J. Chase, I. Foster, M. Milenkovic Presented by Dionysis Logothetis.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
Securing the Grid & other Middleware Challenges Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer.
GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.
GRID ANATOMY Advanced Computing Concepts – Dr. Emmanuel Pilli.
Super Computing 2000 DOE SCIENCE ON THE GRID Storage Resource Management For the Earth Science Grid Scientific Data Management Research Group NERSC, LBNL.
Tool Integration with Data and Computation Grid “Grid Wizard 2”
Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004 GridLab 2003/4 „Steady leadership in changing times!” Jarek Nabrzyski Project Coordinator.
David Foster LCG Project 12-March-02 Fabric Automation The Challenge of LHC Scale Fabrics LHC Computing Grid Workshop David Foster 12 th March 2002.
PROGRESS: GEW'2003 Using Resources of Multiple Grids with the Grid Service Provider Michał Kosiedowski.
Current Globus Developments Jennifer Schopf, ANL.
Tutorial on Science Gateways, Roma, Catania Science Gateway Framework Motivations, architecture, features Riccardo Rotondo.
ACGT Architecture and Grid Infrastructure Juliusz Pukacki ‏ EGEE Conference Budapest, 4 October 2007.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Workload Management Workpackage
The Cactus Team Albert Einstein Institute
Presentation transcript:

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab’s GAT Grid Application Toolkit Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Overview GridLab Intro Goals Architecture API Current status Roadmap Some success stories

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab Project Funded by the EU (5+ M€), January 2002 – December 2004 Application and Testbed oriented Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...: resource management (GRMS), data management, monitoring, adaptive components, mobile user support, security services, portals,... and test them on a real testbed with real applications

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab Members n PSNC (Poznan) - coordination n AEI (Potsdam) n ZIB (Berlin) n Univ. of Lecce n Cardiff University n Vrije Univ. (Amsterdam) n SZTAKI (Budapest) n Masaryk Univ. (Brno) n NTUA (Athens) Sun Microsystems Compaq (HP) n ANL (Chicago, I. Foster) n ISI (LA, C.Kesselman) n UoWisconsin (M. Livny) collaborating with: Users! EU Astrophysics Network, DFN TiKSL/GriKSL NSF ASC Project other Grid projects Globus, Condor, GrADS, PROGRESS, GriPhyn/iVDGL, Most of the other European Grid Projects (GRIDSTART) GWEN

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab Aims Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community), all the other potential grid apps Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 What GridLab isn’t We are not developing low level Grid infrastructure, Addressing Grids and P2P We do not want to repeat work which has already been done (want to incorporate and assimilate it …) Globus APIs, OGSA, ASC Portal (GridSphere/Orbiter), GPDK, GridPort, DataGrid, GriPhyn,...

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab End User Requirements Application oriented environment, Applications on one or more virtual organisations, Flexible, easy-to-use, simple interfaces resources, jobs, and data (including compiling, tracking jobs, cataloguing data), Means to make efficient and effective use of resources, Robustness smart adaptivity, complete control and fail safety are available on all levels, The ability to work in a disconnected environment, Mobile working,

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 What do our users want? Larger computational resources Memory/CPU Faster throughput Cleverer scheduling, configurable scheduling, co-scheduling, exploitation of un-used cycles Easier use of resources Portals, grid application frameworks, information services, mobile devices Remote interaction with simulations and data Notification, steering, visualization, data management Collaborative tools Notification, visualization, video conferencing, portals Dynamic applications, New scenarios Grid application frameworks connecting to services

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab end user requirements From laptops to fully deployed Virtual Organisations, Complexity hidden as much as possible, Collaborative infrastructure, The infrastructure for all classes of applications The infrastructure must provide capabilities to customise choice of service implementation (e.g. using efficiency, reliability, first succeeding, all)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Application Scenarios Dynamic Staging move to faster/cheaper/bigger machine Multiple Universe create clone to investigate steered parameter Automatic Convergence Testing from initial data or initiated during simulation Look Ahead spawn off and run coarser resolution to predict likely future Spawn Independent/Asynchronous Tasks send to cheaper machine, main simulation carries on Application Profiling best machine/queue choose resolution parameters based on queue Dynamic Load Balancing inhomogeneous loads multiple grids Portal User/virtual organisation interface to the grid. Intelligent Parameter Surveys farm out to different machines Make use of Running with management tools such as Condor, Entropia, etc. Scripting thorns (management, launching new jobs, etc) Dynamic use of eg MDS for finding available resources

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Motivation for GAT Why do applications need a framework for using the Grid? Our application developers need a layer between applications and grid infrastructure: Higher level than existing grid APIs, hide complexity, abstract grid functionality through application oriented APIs Insulate against rapid evolution of grid infrastructure Choose between different grid infrastructures Make it possible for grid developers to develop new infrastructures Make it possible for application developers to use and develop for the grid independent of the state of deployment of the grid infrastructure

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Solution... GAT – a layer between apps and emerging grid technologies GridLab testbed/VO Close cooperation between developers and deployers End Users GAT Tool Developers Grid Infrastructure Developers GAT-API Developers

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab Architecture

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab services Software environment for Grid-enabling scientific applications GridLab services, third party services and various core-grid services will be supported by GAT In the advent of the Open Grid Service Architecture (OGSA), GridLab's architecture will revolve around the notion of services, all the GridLab services will be OGSA compliant currently all the services are Web Services based roadmap for Web Services to OGSA transformation is being prepared (3-6 months from now)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab services A primary aim of this project is to produce a GridLab GAT containing a set of high quality services which provide a complete environment for Grid- enabling generic applications GridLab services: implement common (strict) security use common service conneciton protocols (WSDL/OGSA) are built primarily for Globus infrastructure

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 What are the GL services? Authorisation Service (WP6) Adaptive Services (WP7) Data Management Services (WP8) Resource Management System (GRMS) (WP9) Information Services (WP10) Monitoring Services (WP11) Mobile User Support (WP12)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Security (WP6) Security WP focuses right now on the Authorization Service (AS) The main requirement is flexibility The AS is about to provide universal way of defining security policy for the whole Grid, independent of technologies used at lower levels It should be able to implement most security models for Grids and use many different scenarios at the same time

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Adaptive Components (WP7) Adaptive Components Service (ACS) and the Local Adaptive Components (LAC). ACS provides an interface to query the adaptive system. It currently supports calls to: rank resources estimate transfer time estimate usage (of some given metric) LAC uses the monitoring system (shown in blue), to continuously collect data about the resource and applications running on it (load information, queue lengths, network bandwidth to other machines, etc.).

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Data Management Services (WP8) replica catalog prototype was ready at Zakopane meeting data movement/copy service also since Zakopane meeting supports reliable gridftp file transfer is gsi enabled with authentication and delegation scalable and fault-tolerant replica catalog in M12, based on ongoing research at ZIB Soap Host A Host B Host C Host D automatic load-balance, fail-over between replica catalogs external access via SOAP and OGSA internal communication via more efficient protocol (Corba)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Unicore Resource Management (WP9) Job Receiver Jobs Queue Broker Execution Unit Resource Discovery Scheduler Resource Reservation Prediction Unit File Transfer Unit MDS Data Management Executable Repository GRAM Authorization System Adaptive Workflow Manager QoS/SLA Negotiation GRMS Third party systems n Chimera n DataGrid n Crossgrid n...

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab and Condor

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab and GriPhyN

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Information Services (WP10) Client Information Service (OGSA) SOAP over GSI Users Software Services Firewall V.O. C. A. Cluster Job Queues MDS GSI-SASL

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Monitoring System (WP11) Implements GGF’s GMA architecture Fast and robust Small resource usage Can monitor hosts and jobs Can deliver event notifications

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Mobile User Support (WP12) Grid Services Network Environment / Grid Portal Applications User Mobile device

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 The Grid is complex … Monitoring Resource Management Information Security Data Management GLOBUS Application Manager Logging Notification Migration Profiling SOAPWSDLCorbaOGSAOther Other Grid Infrastructure? Application “Is there a better resource I could be using?” UNICORE

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 …need to make it easier to use GAT Application “Is there a better resource I could be using?” GAT_FindResource( ) The Grid

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Grid Application Toolkit The GAT provides functionality through a carefully constructed set of generic high-level APIs, through which an application will be able to call the underlying grid services, Set of application developer APIs for Grid tools, services and software libraries, (and example implementations) that support the development of grid-enabled applications (open source!) Usable from any high level “application” (any generic code, Cactus, Triana, Portals, Scripts, …)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT: AIM Abstract Grid capabilities (services) from the application developer. Application developer concentrates on the functionality as needed by the application. Hide complexity. Provides a layer (buffer zone) between applications and the Grid.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT Goals The GAT provides an API and an associated set of tools which enable end-users and application developers to make easy and flexible use of the Grid, The infrastructure, and in particular the GAT, must allow developers to develop their applications independently of the deployment of grid services, Users must be able to make use of such applications in the absence of a fully-deployed infrastructure.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Cactus/GAT Integration GAT Library Cactus Flesh Thorn CGAT Thorn Physics and Computational Infrastructure Modules Cactus GAT wrappers Additional functionality Build system GridLab Service GridLab Service

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Gridlab Services JXTAWeb Services P2P Subset Protolib JNI P2PS Comms GAP & GAT – P2P GATP2PS GAP Layer Network NS-2 JXTAServe GAT WServe Gridlab Services Other Services

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT Architecture

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT Engine

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 API Call

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Full API Call

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT Init

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GAT Adaptor Interface between GAT Engine and one or more capabilities Translates user requests to appropriate interface syntax for a capability provider Active adaptors change dynamically Includes “security context” Return appropriate error codes Examples OGSA adaptor (provides many capabilities) Globus adaptor (directly talk to gatekeepers) Adaptors for each GridLab service provider “Local” adaptors (GAT_MoveFile => “cp”, GATFindResource => “localhost”)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 The Same Application … Application GAT Application GAT Application GAT Laptop The Grid Super Computer No network! Firewall issues!

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Adaptor Call

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Philosophy Application makes GAT API calls for operations which may be Grid-related. Application links agains the GAT Engine Application runs irrespective of actual underlying infrastructure deployment Engine loads adaptors which are valid in the environment extant when the application starts Adaptors try to do Grid operations on request, on failure another adaptor provided function may be called. Application can thus be compiled, linked and tested without any Grid services Same application executable can run in a full Grid enviroment.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Philosophy The GAT uses whatever underlying Grid infrastructure there is and that people have developed adaptors for, GAT is not about replacing already developed infrastructure, but instead to provide a simple, clear interface which can be used with many different infrastructures. Different versions of Globus Condor Unicore...

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 The GAT Architecture GAT: Grid Application Toolkit API and Toolkit for developing portable Grid applications independently of the underlying Grid infrastructure and available services Implements the GAT-API Used by applications (different languages) GAT Adaptors Connect to capabilities/services GAT Engine Provides the function bindings for the GAT-API

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 API Goals The GAT must support applications written in any language which people write Grid Applications in: C, C++, Fortran, Java, Perl, Python,... The use of the GAT API should be as natural as possible for users of these languages. It must also not require a steep learning curve to move from the API in one language to the API in another language APIs in different languages should be as similar as possible

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 API Roadmap Examine prototype adaptors and define an API based upon the needs expressed in these. (Done) Define in a language independent manner Send out for community review Review API and identify “objects” which the API operates on. Create revised API specification based upon these objects. (Done) Still language independent Use this specification to create language specific bindings.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 API Roadmap Finish API specifications (mostly done, under vendors’s review) Provide “canonical” adaptor which provides all capabilities by some “local” functionality and can be used to provide a test-suite for the API, the GAT- Engine and adaptors Examine prototype design and implementation and produce release-quality code (reference C implementation) providing the actual APIs as opposed to the current APIs, and is deployable on all architectures.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 API Roadmap Use SWIG or some other technology to wrap C implementation and provide the APIs in the other languages Work with Triana and Portal packages to provide a native Java implementation which may be tested against the wrapped reference implementation Use GAT to Grid-enable as large a class of applications as possible already in discussions with lots of application groups Review ! Once there is a fully working system we can't just sit back and smile. There are bound to be problems with provided functionality and there will be feature requests Need an API committee to control additions

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Further Information Prototype in CVS Two documents Requirements Initial requirements based upon end-user and application developer requirements Technical Specification Based on requirements from all work packages and a technology review

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Further information Two API specification Documents: Non object-based http: // 1-GAS-0003.APISpecification.pdf Object-based http: // 1-GAS-0004.ObjectBasedAPISpecification.pdf Mailing List Open subscription

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Migration Scenario Application migrates beacause of bad performance The Goal: Involve all the WPs! GAT application Portal GRMS Adaptive Monitoring GIS Mobile user support Security Data mgmt Testbed

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Migration Scenario

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Migration Scenario

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Migration Scenario

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Migration Scenario

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 The „behind the scenes” goal of GL „Let’s make the most advanced grid in the world” Michael Russell, AEI...and that’s what we do...

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Global Grid Testbed Collaboration (GGTC) Driven by GGF APPS and GridLab testbed and applications Whole testbed constructed very swiftly (few weeks) 5 continents: North America, Europe, Asia, Africa, Australia Over 14 countries, including: China, Japan, Singapore, S.Korea, Egypt, Australia, Canada, Germany, UK, Netherlands, Czech, Hungary, Poland, USA About 70 machines, with thousands of processors (~7500) Many hardware types, including PS2, IA32, IA64, MIPS, IBM Power, Alpha, Hitachi/PPC, Sparc Many OSs, including Linux, Irix, AIX, OSF, True64, Solaris, Hitachi Many different organizations (big centers/individuals) All ran same Grid infrastructure! (Globus)

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Global Grid Testbed Collaboration

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Bandwidth Challenge: Highest Performing Application Distributed simulations using Cactus, Globus and Visapult With John Shalf/LBL and others 16.8 Gigabits/second scinet.supercomp.org/bwc Six sites: USA/Dutch/Czech/Poland

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 TFM Task Farming on the Grid TFM implemented in Cactus GAT (GRAM, GRMS) used for starting remote TFMs Designed for the Grid Tasks can be anything fork/exec

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Task Farming Motivation Requested by local physics group Parameter surveys, e.g. looking for critical phenomena in gravitational wave collapse by varying amplitude, testing different formalisms of Einstein Equations for evolving same initial data Scenario is inherently quite robust and fault tolerant Good migration path to the Grid Start easy (not too much Grid!), task farm across local homogeneous workstations and on single supercomputers. Use public keys first, then test standard Grid infrastructure Use of GAT then means users can start testing GridLab services (should still work for them if services not ready) CGAT team can then test real physics runs using wider Grid and GridLab services.

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Task Farming on the Grid Generic Part Application Specific

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 Grid-xclock Simple application for testing and debugging. xclock is standard X utility, run on any machine with X installed Requires: oxclock binary oX libraries oTo display remotely, need to open outgoing ports from machine it is running on to machine displaying

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 SC2002, Baltimore Varied applications deployed of the GGTC testbed Cactus Black Hole Simulations ASC GridLab Portal Smith-Waterman Nimrod-G GridLab Task Farming scenario Visapult Highlights GGTC won 2 of the 3 HPC Awards Won (with Visapult/LBL group) Bandwidth Challenge $2000 prize money to UNICEF childrens fund

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 More info / summary Check the GridLab tutorials available at the Web Bring your application and test it with the GAT and our services!

Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab Success Stories