J. Hanton - P. Herquet - F. Lequeux - A. Romeyer1 CONDOR-G Installation July 2004 : one independent PC for Grid FTP as a client to UCL August 2004 : complete.

Slides:



Advertisements
Similar presentations
/ 1 N. Williams Grid Middleware Experiences Nadya Williams OCI Grid Computing, University of Zurich
Advertisements

CSF4 Meta-Scheduler Tutorial 1st PRAGMA Institute Zhaohui Ding or
IVOA Interoperablity meeting KyotoMasahiro Tanaka (NAOJ) 1 JVO use of Globus Toolkit Masahiro Tanaka (NAOJ)
Community Grids Lab1 CICC Project Meeting VOTable Developed VotableToSpreadsheet Service which accepts VOTable file location as an input, converts to Excel.
Globus Workshop at CoreGrid Summer School 2006 Dipl.-Inf. Hamza Mehammed Leibniz Computing Centre.
GridPP July 2003Stefan StonjekSlide 1 SAM middleware components Stefan Stonjek University of Oxford 7 th GridPP Meeting 02 nd July 2003 Oxford.
SAM-Grid Status Core SAM development SAM-Grid architecture Progress Future work.
Andrew McNab - Manchester HEP - 17 September 2002 Putting Existing Farms on the Testbed Manchester DZero/Atlas and BaBar farms are available via the Testbed.
Condor Project Computer Sciences Department University of Wisconsin-Madison Eager, Lazy, and Just-in-Time.
NGS computation services: API's,
Generic MPI Job Submission by the P-GRADE Grid Portal Zoltán Farkas MTA SZTAKI.
WS-JDML: A Web Service Interface for Job Submission and Monitoring Stephen M C Gough William Lee London e-Science Centre Department of Computing, Imperial.
Legacy code support for commercial production Grids G.Terstyanszky, T. Kiss, T. Delaitre, S. Winter School of Informatics, University.
MyProxy Guy Warner NeSC Training.
Bob Jones – Project Architecture - 1 March n° 1 Information & Monitoring Services Antony Wilson WP3, RAL
LHCb(UK) Meeting Glenn Patrick1 LHCb Grid Activities in UK LHCb(UK) Meeting Cambridge, 10th January 2001 Glenn Patrick (RAL)
John Kewley CCLRC Daresbury Laboratory NW-GRID Training Event 26 th January 2007 GROWL Scripts and Web Services John Kewley Grid Technology Group E-Science.
1 Automated SFTP Windows and SUN Linux and SUN. 2 Vocabulary  Client = local=the machine generating the SFTP request  Server = remote = the machine.
FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Giuseppe Platania INFN Catania First EELA.
Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.
Evaluation of the Globus Toolkit: Status Roberto Cucchi – INFN Cnaf Antonia Ghiselli – INFN Cnaf Giuseppe Lo Biondo – INFN Milano Francesco Prelz – INFN.
CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.
NIKHEF Testbed 1 Plans for the coming three months.
1 Concepts of Condor and Condor-G Guy Warner. 2 Harvesting CPU time Teaching labs. + Researchers Often-idle processors!! Analyses constrained by CPU time!
A Computation Management Agent for Multi-Institutional Grids
Condor and GridShell How to Execute 1 Million Jobs on the Teragrid Jeffrey P. Gardner - PSC Edward Walker - TACC Miron Livney - U. Wisconsin Todd Tannenbaum.
Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.
CMS HLT production using Grid tools Flavia Donno (INFN Pisa) Claudio Grandi (INFN Bologna) Ivano Lippi (INFN Padova) Francesco Prelz (INFN Milano) Andrea.
USING THE GLOBUS TOOLKIT This summary by: Asad Samar / CALTECH/CMS Ben Segal / CERN-IT FULL INFO AT:
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
1 Workshop 20: Teaching a Hands-on Undergraduate Grid Computing Course SIGCSE The 41st ACM Technical Symposium on Computer Science Education Friday.
K.Harrison CERN, 23rd October 2002 HOW TO COMMISSION A NEW CENTRE FOR LHCb PRODUCTION - Overview of LHCb distributed production system - Configuration.
Workload Management Massimo Sgaravatto INFN Padova.
Grids and Globus at BNL Presented by John Scott Leita.
Zach Miller Computer Sciences Department University of Wisconsin-Madison What’s New in Condor.
Vladimir Litvin, Harvey Newman Caltech CMS Scott Koranda, Bruce Loftis, John Towns NCSA Miron Livny, Peter Couvares, Todd Tannenbaum, Jamie Frey Wisconsin.
Track 1: Cluster and Grid Computing NBCR Summer Institute Session 2.2: Cluster and Grid Computing: Case studies Condor introduction August 9, 2006 Nadya.
The Glidein Service Gideon Juve What are glideins? A technique for creating temporary, user- controlled Condor pools using resources from.
December 8 & 9, 2005, Austin, TX SURA Cyberinfrastructure Workshop Series: Grid Technology: The Rough Guide Grid Technology: The Rough Guide Grid Building.
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
Condor Tugba Taskaya-Temizel 6 March What is Condor Technology? Condor is a high-throughput distributed batch computing system that provides facilities.
03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.
Alain Romeyer - 15/06/20041 CMS farm Mons Final goal : included in the GRID CMS framework To be involved in the CMS data processing scheme.
Grids and Portals for VLAB Marlon Pierce Community Grids Lab Indiana University.
Computational grids and grids projects DSS,
COMP3019 Coursework: Introduction to GridSAM Steve Crouch School of Electronics and Computer Science.
1 st December 2003 JIM for CDF 1 JIM and SAMGrid for CDF Mòrag Burgon-Lyon University of Glasgow.
Part 6: (Local) Condor A: What is Condor? B: Using (Local) Condor C: Laboratory: Condor.
Grid job submission using HTCondor Andrew Lahiff.
July 28' 2011INDIA-CMS_meeting_BARC1 Tier-3 TIFR Makrand Siddhabhatti DHEP, TIFR Mumbai July 291INDIA-CMS_meeting_BARC.
Open Science Grid OSG CE Quick Install Guide Siddhartha E.S University of Florida.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
APST Internals Sathish Vadhiyar. apstd daemon should be started on the local resource Opens a port to listen for apst client requests Runs on the host.
Proposal for a IS schema Massimo Sgaravatto INFN Padova.
Pilot Factory using Schedd Glidein Barnett Chiu BNL
August 30, 2002Jerry Gieraltowski Launching ATLAS Jobs to either the US-ATLAS or EDG Grids using GRAPPA Goal: Use GRAPPA to launch a job to one or more.
© Geodise Project, University of Southampton, Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Grid2Win : gLite for Microsoft Windows Roberto.
Grid Compute Resources and Job Management. 2 Grid middleware - “glues” all pieces together Offers services that couple users with remote resources through.
The Gateway Computational Web Portal Marlon Pierce Indiana University March 15, 2002.
JSS Job Submission Service Massimo Sgaravatto INFN Padova.
Open Science Grid Build a Grid Session Siddhartha E.S University of Florida.
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
TANYA LEVSHINA Monitoring, Diagnostics and Accounting.
Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA Grid2Win : gLite for Microsoft Windows Elisa Ingrà - INFN.
Gri2Win: Porting gLite to run under Windows XP Platform
Grid2Win Porting of gLite middleware to Windows XP platform
Gri2Win: Porting gLite to run under Windows XP Platform
Alice Software Demonstration
Presentation transcript:

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer1 CONDOR-G Installation July 2004 : one independent PC for Grid FTP as a client to UCL August 2004 : complete installation of Globus 3.2 on the farm GridFTP server -> another picture to download Interface with CONDOR for batch submission Local test with the independent PC Globus-url-copy –v gsiftp://cms01.umh.ac.be/tmp/newlab_we_are_ready.jpg file ${HOME}/newlab_we_are_ready.jpg

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer2 Cluster architecture Server Router Computer … 1 Gb/s 100 Mb/s Outer world Raid disk (2.4 TB) OS : Redhat CERN Cms Cms cms01.umh.ac.be Static IP XXX Independent PC CONDOR Globus

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer3 Globus setup On both the indep. PC and CMS01 public machine : Configure the GridFTP server on port 2811 Configure the gatekeeper service on port 2119 Setting up the grid-mapfile : Setting up the grim port types : "/C=BE/O=BELGRID/OU=TESTBED/OU=umh.ac.be/CN=cmsuser" cmsuser "/C=BE/O=BELGRID/OU=TESTBED/OU=localdomain/CN=cmsuser" cmsuser "/C=BE/O=BELGRID/OU=TESTBED/OU=fynu.ucl.ac.be/CN=Alain NINANE" cmsuser d_job/ManagedJobPortType ed_job/ManagedJobPortType

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer4 Globus setup… Lets test it : Nothing strange in the globus-gatekeeper.log Solution (only a trick…) : remove the gsi-authz.conf in /etc/grid-security Build the condor scheduler in Globus gpt-build scheduler-condor-3.2-src_bundle.tar.gz gcc32dbg gpt-postinstall Globus-job-run cms01.umh.ac.be /bin/date Gram Job submission failed because data transfer to the server failed (error code 10) globus-job-run cms01.umh.ac.be /bin/date Thu Aug 19 10:12:34 CEST 2004 globus-job-run cms01.umh.ac.be /bin/date Thu Aug 19 10:12:16 CEST 2004 Local test Remote test

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer5 Test of CONDOR-G Start with a CONDOR example : io.c CONDOR.cmd file : #################### ## ## Test Condor command file ## #################### universe = globus globusscheduler = cms01.umh.ac.be/jobmanager-condor executable = io.remote output = io.out error = io.err log = io.log requirements = CMSFARM=?=True arguments = 200 queue Indep. PC CONDOR + Globus eth0 eth1 CMS01 CONDOR + Globus eth0 CMS02, CMS03 CONDOR Globus CONDOR

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer6 Test of CONDOR-G… Launch the job from the indep. PC : condor_submit io.cmd condor_q -globus -- Submitter: cms-test.umh.ac.be : : cms-test.umh.ac.be ID OWNER STATUS MANAGER HOST EXECUTABLE cmsuser UNSUBMITTED condor cms01.umh.ac.be /tmp/Scratch/examp condor_q -globus -- Submitter: cms-test.umh.ac.be : : cms-test.umh.ac.be ID OWNER STATUS MANAGER HOST EXECUTABLE cmsuser PENDING condor cms01.umh.ac.be /tmp/Scratch/examp condor_q -globus -- Submitter: cms-test.umh.ac.be : : cms-test.umh.ac.be ID OWNER STATUS MANAGER HOST EXECUTABLE cmsuser ACTIVE condor cms01.umh.ac.be /tmp/Scratch/examp

J. Hanton - P. Herquet - F. Lequeux - A. Romeyer7 Test of CONDOR-G… On cms01 : condor_q -- Submitter: cms01.umh.ac.be : : cms01.umh.ac.be ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD cmsuser 8/19 10: :00:51 R data Lumi2 1 jobs; 0 idle, 1 running, 0 held condor_q -r -- Submitter: cms01.umh.ac.be : : cms01.umh.ac.be ID OWNER SUBMITTED RUN_TIME HOST(S) cmsuser 8/19 10: :01:00 cms02 Job is running on cms02 Rem : the test has also been done with CMS reconstruction job