Portals Team GridSphere and the GridLab Project Jason Novotny Michael Russell Oliver Wehrens Albert.

Slides:



Advertisements
Similar presentations
PRAGMA BioSciences Portal Raj Chhabra Susumu Date Junya Seo Yohei Sawai.
Advertisements

W w w. h p c - e u r o p a. o r g HPC-Europa Portal: Uniform Access to European HPC Infrastructure Ariel Oleksiak Poznan Supercomputing.
Jason Novotny GridSphere / Portlets GridSphere Status Update Jason Novotny Lawrence Berkeley National Laboratory Jason Novotny,
Using the Collaborative Tools in NEESgrid Charles Severance University of Michigan.
1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
GridLab Enabling Applications on the Grid Jarek Nabrzyski et al. Poznań Supercomputing and Networking.
The Cactus Portal A Case Study in Grid Portal Development Michael Paul Russell Dept of Computer Science The University of Chicago
The CrossGrid project Juha Alatalo Timo Koivusalo.
Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.
1-2.1 Grid computing infrastructure software Brief introduction to Globus © 2010 B. Wilkinson/Clayton Ferner. Spring 2010 Grid computing course. Modification.
SC 2003 Demo, NCSA booth GridLab Project Funded by the EU (5+ M€), January 2002 – December 2004 Application and Testbed oriented Cactus Code, Triana Workflow,
W4: Grid Portals Building Web-enabled End-User Environments for accessing Grid Services By Michael Paul Russell Dept Computer Science.
GridLab & Cactus Joni Kivi Maarit Lintunen. GridLab  A project funded by the European Commission  The project was started in January 2002  Software.
Jarek Nabrzyski Crossgrid Meeting in Poznań, 30 July 2003 GridLab’s GAT Grid Application Toolkit Jarek Nabrzyski et al. Poznań Supercomputing.
Kelly Davis GAT: Grid Application Toolkit Kelly Davis AEI-MPG.
GridSphere for GridLab A Grid Application Server Development Framework By Michael Paul Russell Dept Computer Science University.
Globus Computing Infrustructure Software Globus Toolkit 11-2.
W w w. h p c - e u r o p a. o r g The HPC-Europa project and GridSphere Dawid Szejnfeld Poznan Supercomputing.
SUN HPC Consortium, Heidelberg 2004 Grid(Lab) Resource Management System (GRMS) and GridLab Services Krzysztof Kurowski Poznan Supercomputing and Networking.
Grid Computing, B. Wilkinson, a.1 Grid Portals.
Grid Workflow within Triana Ian Wang Cardiff University.
XCAT Science Portal Status & Future Work July 15, 2002 Shava Smallen Extreme! Computing Laboratory Indiana University.
General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.
Holding slide prior to starting show. A Grid-based Problem Solving Environment for GECEM Maria Lin and David Walker Cardiff University Yu Chen and Jason.
Portals Team GridSphere: A Grid Portal Framework An Overview Ian Kelley Jason Novotny
A Lightweight Platform for Integration of Resource Limited Devices into Pervasive Grids Stavros Isaiadis and Vladimir Getov University of Westminster
GridLab A Grid Application Toolkit and Testbed IST Jarek Nabrzyski GridLab Project Coordinator Poznań.
5th GridLab Review, 8 April 2005, Amsterdam WP12: Access for mobile users What's new in mobile user support Piotr Grabowski, Bartek Lewandowski
CoG Kit Overview Gregor von Laszewski Keith Jackson.
WP9 Resource Management Current status and plans for future Juliusz Pukacki Krzysztof Kurowski Poznan Supercomputing.
Portals & Portlets 2003, 14 July 2003 The GridLab Portal Portals & Portlets 2003 Albert Einstein Institute Jason Novotny
GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.
Cactus Project & Collaborative Working Gabrielle Allen Max Planck Institute for Gravitational Physics, (Albert Einstein Institute)
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
Through the development of advanced middleware, Grid computing has evolved to a mature technology in which scientists and researchers can leverage to gain.
Applications for the Grid Here at GGF1: Gabrielle Allen, Thomas, Dramlitsch, Gerd Lanfermann, Thomas Radke, Ed Seidel Max Planck Institute for Gravitational.
GridLab: A Grid Application Toolkit and Testbed Jarosław Nabrzyski GridLab Project Manager Poznań Supercomputing and Networking Center, Poland
GridLab: A Grid Application Toolkit and Testbed
GridLab Review - WP4 WP4 - Grid Portals GridLab Review Michael Russell Albert Einstein Institute.
Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.
General Intro to GridLab Jarek Nabrzyski et al. Poznań Supercomputing and Networking Center.
April 1st, The ASC- GridLab Portal Edward Seidel, Michael Russell, Gabrielle Allen, and the rest of the team Max Plank Institut für Gravitationsphysik.
Introduction to Grid Computing Ed Seidel Max Planck Institute for Gravitational Physics
Tools for collaboration How to share your duck tales…
1 Grid Portal for VN-Grid Cu Nguyen Phuong Ha. 2 Outline Some words about portals in principle Overview of OGCE GridPortlets.
Rob Allan Daresbury Laboratory A Web Portal for the National Grid Service Xiaobo Yang, Dharmesh Chohan, Xiao Dong Wang and Rob Allan CCLRC e-Science Centre,
Holding slide prior to starting show. A Portlet Interface for Computational Electromagnetics on the Grid Maria Lin and David Walker Cardiff University.
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.
Cactus/TIKSL/KDI/Portal Synch Day. Agenda n Main Goals:  Overview of Cactus, TIKSL, KDI, and Portal efforts  present plans for each project  make sure.
GridLab WP-2 Cactus GAT (CGAT) Ed Seidel, AEI & LSU Co-chair, GGF Apps RG, Gridstart Apps TWG Gabrielle Allen, Robert Engel, Tom Goodale, *Thomas Radke.
GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.
Distributed Data for Science Workflows Data Architecture Progress Report December 2008.
Development of e-Science Application Portal on GAP WeiLong Ueng Academia Sinica Grid Computing
Hiding Grid Complexity Behind SSH Session Server framework Tomasz Kuczyński (1,2) 1) Poznan Supercomputing and Networking Center.
WebFlow High-Level Programming Environment and Visual Authoring Toolkit for HPDC (desktop access to remote resources) Tomasz Haupt Northeast Parallel Architectures.
VisPortal Project developer’s experience C.E.Siegerist, J. Shalf, E.W. Bethel NERSC/LBNL Visualization Group T.J. Jankun-Kelley, O. Kreylos, K.L. Ma CIPIC/UC.
AHM, Aug-30-Sept-2, 2004 Virtual Research in the UK: Advanced Portal Services Mark Baker and Hong Ong Distributed Systems Group University of Portsmouth.
GridLab Resource Management System (GRMS) Jarek Nabrzyski GridLab Project Coordinator Poznań Supercomputing and.
Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004 GridLab 2003/4 „Steady leadership in changing times!” Jarek Nabrzyski Project Coordinator.
© Copyright AARNet Pty Ltd PRAGMA Update & some personal observations James Sankar Network Engineer - Middleware.
Dynamic Grid Computing: The Cactus Worm The Egrid Collaboration Represented by: Ed Seidel Albert Einstein Institute
Holding slide prior to starting show. Lessons Learned from the GECEM Portal David Walker Cardiff University
Tutorial on Science Gateways, Roma, Catania Science Gateway Framework Motivations, architecture, features Riccardo Rotondo.
ACGT Architecture and Grid Infrastructure Juliusz Pukacki ‏ EGEE Conference Budapest, 4 October 2007.
The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.
Shaowen Wang 1, 2, Yan Liu 1, 2, Nancy Wilkins-Diehr 3, Stuart Martin 4,5 1. CyberInfrastructure and Geospatial Information Laboratory (CIGI) Department.
Cactus Project & Collaborative Working
Shaowen Wang1, 2, Yan Liu1, 2, Nancy Wilkins-Diehr3, Stuart Martin4,5
Grid Computing AEI Numerical Relativity Group has access to high-end resources in over ten centers in Europe/USA They want: Bigger simulations, more simulations.
Presentation transcript:

Portals Team GridSphere and the GridLab Project Jason Novotny Michael Russell Oliver Wehrens Albert Einstein Institute

The State of Grid Computing Access to Grid services and resources still difficult for end users Current API’s and tools are still very primitive Grid middleware and tools are constantly evolving making it difficult to provide reliable, robust infrastructure Ultimately end users want to be hidden from technical or implementation specific details of Grids Growing infrastructure complexity means diminished usability Many projects focusing on a narrow range of services for a very specific set of users There is a real need for higher level tools and grid computing environments

GridLab Project Funded by the EU (5+ M€), January 2002 – December 2004 Application and Testbed oriented Cactus Code, Triana Workflow, all the other applications that want to be Grid- enabled Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...: resource management (GRMS), data management, monitoring, adaptive components, mobile user support, security services, A portal... and test them on a real testbed with real applications

GridLab Services Resource Management Services Resource Brokering Job Submission Usage Policy Management Data Management Services Data Replication Date Movement Visualization Services Monitoring and Adaptive Services Application Performance Monitoring Resource Utilization The ability to alter applications at runtime Information Services GIIS/GRIS Based Testbed Management Services Form the basis of the GridLab Administrative Portlets Notification Services… and many more!

GridLab VO PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens) Sun Microsystems Compaq (HP) ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny) collaborating with: Users! EU Astrophysics Network, DFN TiKSL/GriKSL NSF ASC Project other Grid projects Globus, Condor, GrADS, PROGRESS, GriPhyn/iVDGL, CrossGrid and all the other European Grid Projects (GRIDSTART) other...

2004 Michael Russell 6 GridLab Goals Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community), all the other potential grid apps Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

2004 Michael Russell 7 What Do Our Users Need? Application oriented environment Flexible, easy-to-use, simple interfaces Efficient and effective use of resources Robustness, fail-safety, adapability The ability to work in distributed teams Support for mobile working environments

2004 Michael Russell 8 What Do Our Users Want? Larger computational resources Memory/CPU Faster throughput Cleverer scheduling, configurable scheduling, co-scheduling, exploitation of un- used cycles Easier use of resources Portals, grid application frameworks, information services, mobile devices Remote interaction with simulations and data Notification, steering, visualization, data management Collaborative tools Notification, visualization, video conferencing, portals Dynamic applications, New scenarios Grid application frameworks connecting to services

2004 Michael Russell 9 Many Application Scenarios! Dynamic Staging move to faster/cheaper/bigger machine Multiple Universe create clone to investigate steered parameter Automatic Convergence Testing from initial data or initiated during simulation Look Ahead spawn off and run coarser resolution to predict likely future Spawn Independent/Asynchronous Tasks send to cheaper machine, main simulation carries on Application Profiling best machine/queue choose resolution parameters based on queue Dynamic Load Balancing inhomogeneous loads multiple grids Portal User/virtual organisation interface to the grid. Intelligent Parameter Surveys farm out to different machines Make use of Running with management tools such as Condor, Entropia, etc. Scripting thorns (management, launching new jobs, etc) Dynamic use of eg MDS for finding available resources

Portal Bridges Users and Services

Virtual Organizations A Virtual Organization (VO) consists of resources, services, and people collaborating across institutional, geographical, and political boundaries. Grid Portals provide a gateway for linking all of these elements together Grid Portals hide the complexity of the underlying Grid infrastructure that make VOs possible. Furthermore, Grid Portals can provide value-added services with Web interfaces tailored to the specific applications that are most important to particular VO.

Grid Portals in the past... A Portal is only as good as the underlying deployed infrastructure Portlet development often involves debugging underlying middleware Often difficult and hard to maintain glue code must be written connecting the portal to Grid services, due to lack of/evolving standards. Most portals are stovepipe solutions that provide a complete solution with very little customization capabilities. Separation of presentation and login Generally hard coding of underlying Grid infrastructure details and the codebase Lack of real usability has made it difficult to test and evaluate user interfaces. Web application development still remains a tedious task with little in the way of reusable components, forcing developers to constantly “re- invent” the wheel.

Portal standards JSR 168 Portlet API ratified August 2003 Similar to Servlet API in providing reusable web applications Ratified by vendors including BEA, Sun, IBM, Oracle, Plumtree and others... WSRP (Web Services for Remote Portlets) ratified by OASIS committee Specifies how web services can be consumed by standards compliant portals Java Server Faces ratified Specifies an event based user interface for web presentation development

GridSphere Portal Framework GridLab Portal is developed using the GridSphere Portal Framework GridSphere provides a generic JSR 168 compliant portlet container The GridSphere portal framework provides an architecture for deploying “pluggable” web applications using the Portlet model. Provides a development environment for easily creating new portlet applications Offers a core set of portlets for the management of portlets, users, groups and layouts. A successful open source development model: Active and responsive mail list CVS access Bugtracker

GridLab Portlets The GridLab portal is an instance of GridSphere customized for the users and requirements of the GridLab project GridLab portlets can easily “plug” into GridSphere portlet container-- and hopefully any other portlet container. GridLab portlets provide access to GridLab services and can be modified/reused by other Grid projects. Core aspects of grid portlets include security, job execution, file management/data transfer and providing resource information

GridLab Portlets (cont.) Currently developed portlets to support GridLab: General job submission portlet Supports GRMS Job Broker service developed by Work Package 9 Supports Globus job submission Soon to come is GSI-SSH and Condor submission General credential management portlet Uses Myproxy for the secure storage and management of users credentials. Allows users to selectively retrieve and use credentials for performing Grid operations. Monitoring portlet demonstrates monitoring service developed by Work package 11

External Collaborations Working to support GridSphere for the general Grid portal community: Albert-Einstein-institute in Berlin & LSU to develop portlets for studying numerical relativity. Geon earth sciences, microscopy portal and others at SDSC Canadian National research Council (NRC) to develop grid data transfer portlets using GridFTP SZTAKI research center in Budapest to use GridSphere to support users of SuperGrid and ClusterGrid projects Virtual Observatory project in Australia PSNC in Poland to support ClusterIX project Many more: Interested in promoting GridSphere, grid computing and portlets to more communities! Please us if interested!