Andrew McNab - Globus Distribution for Testbed 1 Status of the Globus Distribution for Testbed 1 Andrew McNab, University of Manchester

Slides:



Advertisements
Similar presentations
Status Report University of Bristol 3 rd GridPP Collaboration Meeting 14/15 February, 2002Marc Kelly University of Bristol 1 Marc Kelly University of Bristol.
Advertisements

Andrew McNab - Manchester HEP - 17 September 2002 Putting Existing Farms on the Testbed Manchester DZero/Atlas and BaBar farms are available via the Testbed.
Andrew McNab - Manchester HEP - 24 May 2001 WorkGroup H: Software Support Both middleware and application support Installation tools and expertise Communication.
Andrew McNab - Manchester HEP - 2 May 2002 Testbed and Authorisation EU DataGrid Testbed 1 Job Lifecycle Software releases Authorisation at your site Grid/Web.
NorduGrid Grid Manager developed at NorduGrid project.
CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.
German Cancio – WP4 developments Partner Logo WP4-install plans WP6 meeting, Paris project conference
NIKHEF Testbed 1 Plans for the coming three months.
Andrew McNab - Manchester HEP - 29/30 March 2001 gridmapdir patch Overview of the problem Constraints from local systems Outline of how it works How to.
Andrew McNab - EDG Access Control - 14 Jan 2003 EU DataGrid security with GSI and Globus Andrew McNab University of Manchester
Adding scalability to legacy PHP web applications Overview Mario A. Valdez-Ramirez.
CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea.
USING THE GLOBUS TOOLKIT This summary by: Asad Samar / CALTECH/CMS Ben Segal / CERN-IT FULL INFO AT:
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Andrew McNab - Manchester HEP - 6 November Old version of website was maintained from Unix command line => needed (gsi)ssh access.
Status of Globus activities within INFN Massimo Sgaravatto INFN Padova for the INFN Globus group
INFN-GRID Globus evaluation Massimo Sgaravatto INFN Padova for the INFN Globus group
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Status of Globus activities within INFN (update) Massimo Sgaravatto INFN Padova for the INFN Globus group
Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.
C. Loomis – Testbed: Status… – Sep. 5, 2002 – 1 Testbed: Status & Plans Charles Loomis (CNRS) Sept. 5, th Project Conference (Budapest)
Zach Miller Condor Project Computer Sciences Department University of Wisconsin-Madison Flexible Data Placement Mechanisms in Condor.
Andrew McNab - Manchester HEP - 26 June 2001 WG-H / Support status Packaging / RPM’s UK + EU DG CA’s central grid-users file grid “ping”
Don Quijote Data Management for the ATLAS Automatic Production System Miguel Branco – CERN ATC
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
Andrew McNab - Manchester HEP - 5 July 2001 WP6/Testbed Status Status by partner –CNRS, Czech R., INFN, NIKHEF, NorduGrid, LIP, Russia, UK Security Integration.
GRAM5 - A sustainable, scalable, reliable GRAM service Stuart Martin - UC/ANL.
Grid and Cloud Computing Dr. Guy Tel-Zur. Today’s agenda UNICORE (see a separate presentation) AWS + Python (Boto) – ideas for projects… Hadoop (see a.
Grid Workload Management Massimo Sgaravatto INFN Padova.
MAGDA Roger Jones UCL 16 th December RWL Jones, Lancaster University MAGDA  Main authors: Wensheng Deng, Torre Wenaus Wensheng DengTorre WenausWensheng.
C. Loomis – Integration Status- October 31, n° 1 Integration Status October 31, 2001
Andrew McNabETF Firewall Meeting, NeSC, 5 Nov 2002Slide 1 Firewall issues for Globus 2 and EDG Andrew McNab High Energy Physics University of Manchester.
Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.
Author - Title- Date - n° 1 Partner Logo WP5 Summary Paris John Gordon WP5 6th March 2002.
Globus Replica Management Bill Allcock, ANL PPDG Meeting at SLAC 20 Sep 2000.
First attempt for validating/testing Testbed 1 Globus and middleware services WP6 Meeting, December 2001 Flavia Donno, Marco Serra for IT and WPs.
GRIDS Center Middleware Overview Sandra Redman Information Technology and Systems Center and Information Technology Research Center National Space Science.
INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)
National Center for Supercomputing ApplicationsNational Computational Science Grid Packaging Technology Technical Talk University of Wisconsin Condor/GPT.
GRID Zhen Xie, INFN-Pisa, on DataGrid WP6 meeting1 Globus Installation Toolkit Zhen Xie On behalf of grid-release team INFN-Pisa.
DGC Paris WP2 Summary of Discussions and Plans Peter Z. Kunszt And the WP2 team.
January 26, 2003Eric Hjort HRMs in STAR Eric Hjort, LBNL (STAR/PPDG Collaborations)
UK Grid Meeting Glenn Patrick1 LHCb Grid Activities in UK Grid Prototype and Globus Technical Meeting QMW, 22nd November 2000 Glenn Patrick (RAL)
K. De UTA Grid Workshop April 2002 ATLAS Pre-packaged Kaushik De University of Texas at Arlington.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
INFSO-RI Enabling Grids for E-sciencE Ganga 4 – The Ganga Evolution Andrew Maier.
Andrew McNab - Manchester HEP - 17 September 2002 UK Testbed Deployment Aim of this talk is to the answer the questions: –“How much of the Testbed has.
Andrew McNabGrid in 2002, Manchester HEP, 7 Jan 2003Slide 1 Grid Work in 2002 Andrew McNab High Energy Physics University of Manchester.
German Cancio – WP4 developments Partner Logo WP4-install progress CERN, 19/6/2002 for WP4-install.
Security Middleware 3 June 2004 Security Middleware Current Status – GridSite deployments – Architecture GridPP2 – Web services.
Andrew McNab - Security issues - 17 May 2002 WP6 Security Issues (some personal observations from a WP6 and sysadmin perspective) Andrew McNab, University.
Andrew McNab - Security issues - 4 Mar 2002 Security issues for TB1+ (some personal observations from a WP6 and sysadmin perspective) Andrew McNab, University.
Andrew McNab - Globus Distribution for Testbed 1 Globus Distribution for Testbed 1 Andrew McNab, University of Manchester
Data Management The European DataGrid Project Team
Author - Title- Date - n° 1 Partner Logo WP5 Status John Gordon Budapest September 2002.
Grid Technologies for Distributed Database Services 3D Project Meeting CERN, May 19, 2005 A. Vaniachine (ANL)
15-Feb-02Steve Traylen, RAL WP6 Test Bed Report1 RAL/UK WP6 Test Bed Report Steve Traylen, WP6 PPGRID/RAL, UK
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
Andrew McNab - Manchester HEP - 10 May 2001 EDG / Globus Meeting, 5-7 April “Obtaining an up-to-date understanding of the state of the Globus Toolkit,
Protocols and Services for Distributed Data- Intensive Science Bill Allcock, ANL ACAT Conference 19 Oct 2000 Fermi National Accelerator Laboratory Contributors:
WP1 Status and plans Francesco Prelz, Massimo Sgaravatto 4 th EDG Project Conference Paris, March 6 th, 2002.
LHCb Grid MeetingLiverpool, UK GRID Activities Glenn Patrick Not particularly knowledgeable-just based on attending 3 meetings.  UK-HEP.
Tests at Saclay D. Calvet, A. Formica, Z. Georgette, I. Mandjavidze, P. Micout DAPNIA/SEDI, CEA Saclay Gif-sur-Yvette Cedex.
Testbed Release Criteria and Plans WP8 Loose Cannons & App Representatives.
Current Globus Developments Jennifer Schopf, ANL.
Maite Barroso – WP4 Workshop – 10/12/ n° 1 -WP4 Workshop- Developers’ Guide Maite Barroso 10/12/2002
GridSite status Andrew McNab University of Manchester.
1 April 2, Software Packaging and Releasing Best Practices William Cohen NCSU CSC 591W April 2, 2008.
Release BETA Integration Tool Status Geneva – 1 March 2007
Presentation transcript:

Andrew McNab - Globus Distribution for Testbed 1 Status of the Globus Distribution for Testbed 1 Andrew McNab, University of Manchester

Andrew McNab - Globus Distribution for Testbed 1 Overview u Current release: beta-21 (= Globus 2.0beta1) u “Showstoppers” with Globus 2.0beta1 u Release-24 (= Globus / 2.2beta) u Static and shared versions of Release-24 u What will be in 1.2.x u Going to a pure Globus 2.2 Testbed release u Summary

Andrew McNab - Globus Distribution for Testbed 1 Current release: beta-21 u This is based on Globus 2.0 beta1 released last November. u Beta code only differed from alpha releases (eg alpha-15) in terms of bug fixes. u However, which flavors of each subpackages are available followed Globus recommendations (this mainly affected pgm -> pgm_static) u We stopped building the “empty” Globus subpackages any more (ie that just contain the metadata file and filelist.) u Several bugfixes were introduced during 2002, with subrelease numbers in RPM filename (eg 21b) n See u But, this version has some serious problems when used for Testbed.

Andrew McNab - Globus Distribution for Testbed 1 Showstoppers in Globus 2.0beta1 (beta-24) u GASS Cache problem which affects jobmanager n The cache management code can produce corrupted caches. n This means job submission at high rate (> 5 at the same time!) leads to a significant percentage of job submissions failing (100% in worst case) n Various solutions have been tried. n Current solution uses Condor rewrite for Globus 2.2 backported to 2.0 by Francesco Prelz. u GridFTP transfer time limit n This results in an unintended time limit on transfers of about 21 minutes n Not related to size, just time. u MDS instabilities n makes a hierarchy unusable in practice

Andrew McNab - Globus Distribution for Testbed 1 Release-24 u This is based on Globus from last month (equivalent to one of the Globus 2.2 betas.) u We’ve made some changes to the packaging, to make it simpler to write rpmlists: each package (globus_gss_assist) now has one RPM with binaries, headers, libraries etc. u Globus includes fixes for GASS Cache, GridFTP and MDS showstoppers. u BUT introduces incompatibilities: n Job submission (where cache problem shows up) has new PBS etc script format (in Perl not Bash now!) and RSL no longer compatible with RSL used by WP1 n New GridFTP not backwards compatible. n Some changes to MDS schemas.

Andrew McNab - Globus Distribution for Testbed 1 Static and shared versions of Release-24 u By default, all packages in release-24 were built with shared libraries n makes it easier to update one component in isolation. u However, to test and possibly use fixes from Globus 2.1.3, also built static library versions to be used alongside beta-21 n Install into /opt/staticglobus-24 u However, some MDS packages cannot be built static (they load shared modules) so need to be able to install from shared release n Install into /opt/globus-24 u This means can install alongside beta-21 which is still the official release n Installs into /opt/globus

Andrew McNab - Globus Distribution for Testbed 1 What will be in 1.2.x (1.3.x?) u The Testbed release intended to fix the 3 showstoppers will have: n Beta-21 as in 1.2.2, plus: n Francesco’s backported Condor GASS cache fix for beta-21 n The GridFTP server and globus-url-copy client from release-24 n The MDS from release-24 n Necessary globus-edgconfig updates for the above. n Some EDG packages updated for use with release-24 s GDMP and Replica Manager (relinked) s Replica Catalog and Job Submission (changed MDS schema)

Andrew McNab - Globus Distribution for Testbed 1 Going to a pure Globus 2.2 Testbed release u Obviously this mixture of Globus versions is a hack n will lead to conflicts and inconsistencies of its own n harder to support. u Globus 2.2 fixes what’s wrong with beta-21 but has incompatibilities with it u To use it we need to n make a release-25 built from the released Globus updates n get a working configuration for it (eg get PBS scripts that work) n modify our own packages where the Globus API has changed (eg RSL) u We would then have just one, up to date Globus installation which we can legitimately complain about if it doesn’t work.

Andrew McNab - Globus Distribution for Testbed 1 Summary u Beta-21 (Globus 2.0beta1) is still the official current version u Release-24 (Globus 2.1.3) exists in shared and static forms u A mix of beta-21 and release-24 exists which fixes the showstoppers. u This will be used for 1.2.x (1.3.x?) u We would have to make some changes to our packages to work with Globus updates as released.