Interfacing SCMSWeb with Condor-G – A Joint PRAGMA-Condor Effort.

Slides:



Advertisements
Similar presentations
Resource WG Breakout. Agenda How we will support/develop data grid testbed and possible applications (1 st day) –Introduction of Gfarm (Osamu) –Introduction.
Advertisements

Jaime Frey Computer Sciences Department University of Wisconsin-Madison OGF 19 Condor Software Forum Routing.
Setting up of condor scheduler on computing cluster Raman Sehgal NPD-BARC.
Condor and GridShell How to Execute 1 Million Jobs on the Teragrid Jeffrey P. Gardner - PSC Edward Walker - TACC Miron Livney - U. Wisconsin Todd Tannenbaum.
Condor DAGMan Warren Smith. 12/11/2009 TeraGrid Science Gateways Telecon2 Basics Condor provides workflow support with DAGMan Directed Acyclic Graph –Each.
Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Reliability and Troubleshooting with Condor Douglas Thain Condor Project University of Wisconsin PPDG Troubleshooting Workshop 12 December 2002.
Workload Management Workpackage Massimo Sgaravatto INFN Padova.
GRID Workload Management System Massimo Sgaravatto INFN Padova.
Workload Management Massimo Sgaravatto INFN Padova.
First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova
First ideas for a Resource Management Architecture for Productions Massimo Sgaravatto INFN Padova.
Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
High Throughput Computing with Condor at Purdue XSEDE ECSS Monthly Symposium Condor.
Prof. Heon Y. Yeom Distributed Computing Systems Lab. Seoul National University FT-MPICH : Providing fault tolerance for MPI parallel applications.
Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.
Progress Report Barnett Chiu Glidein Code Updates and Tests (1) Major modifications to condor_glidein code are as follows: 1. Command Options:
Somsak Sriprayoonskul, Nopparat Nopkuat, Putchong Uthayopas, Sugree Phatanapherom ThaiGrid, Thailand Cindy Zheng, SDSC.
STAR scheduling future directions Gabriele Carcassi 9 September 2002.
Campus Grids Report OSG Area Coordinator’s Meeting Dec 15, 2010 Dan Fraser (Derek Weitzel, Brian Bockelman)
Grid Computing I CONDOR.
High Throughput Parallel Computing (HTPC) Dan Fraser, UChicago Greg Thain, UWisc Condor Week April 13, 2010.
Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.
3-2.1 Topics Grid Computing Meta-schedulers –Condor-G –Gridway Distributed Resource Management Application (DRMAA) © 2010 B. Wilkinson/Clayton Ferner.
Grid tool integration within the eMinerals project Mark Calleja.
Experiences with a HTCondor pool: Prepare to be underwhelmed C. J. Lingwood, Lancaster University CCB (The Condor Connection Broker) – Dan Bradley
DataGrid WP1 Massimo Sgaravatto INFN Padova. WP1 (Grid Workload Management) Objective of the first DataGrid workpackage is (according to the project "Technical.
Rochester Institute of Technology Job Submission Andrew Pangborn & Myles Maxfield 10/19/2015Service Oriented Cyberinfrastructure Lab,
Grid job submission using HTCondor Andrew Lahiff.
July 28' 2011INDIA-CMS_meeting_BARC1 Tier-3 TIFR Makrand Siddhabhatti DHEP, TIFR Mumbai July 291INDIA-CMS_meeting_BARC.
Grid Compute Resources and Job Management. 2 Local Resource Managers (LRM)‏ Compute resources have a local resource manager (LRM) that controls:  Who.
Matlab Demo #1 ODE-solver with parameters. Summary Here we will – Modify a simple matlab script in order to split the tasks to be sent to the cluster.
Report from USA Massimo Sgaravatto INFN Padova. Introduction Workload management system for productions Monte Carlo productions, data reconstructions.
TeraGrid Advanced Scheduling Tools Warren Smith Texas Advanced Computing Center wsmith at tacc.utexas.edu.
Condor-G A Quick Introduction Alan De Smet Condor Project University of Wisconsin - Madison.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
Review of Condor,SGE,LSF,PBS
SC2008 (11/19/2008) Resources Group Pacific Rim Application and Grid Middleware Assembly Reports.
Pilot Factory using Schedd Glidein Barnett Chiu BNL
FermiGrid School Steven Timm FermiGrid School FermiGrid 201 Scripting and running Grid Jobs.
Grid Compute Resources and Job Management. 2 Job and compute resource management This module is about running jobs on remote compute resources.
AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.
Grid Compute Resources and Job Management. 2 Grid middleware - “glues” all pieces together Offers services that couple users with remote resources through.
The Gateway Computational Web Portal Marlon Pierce Indiana University March 15, 2002.
JSS Job Submission Service Massimo Sgaravatto INFN Padova.
STAR Scheduling status Gabriele Carcassi 9 September 2002.
4/9/ 2000 I Datagrid Workshop- Marseille C.Vistoli Wide Area Workload Management Work Package DATAGRID project Parallel session report Cristina Vistoli.
Digital Media SEAGF Somsak Sriprayoonsakul Thai National Grid Center
HTCondor’s Grid Universe Jaime Frey Center for High Throughput Computing Department of Computer Sciences University of Wisconsin-Madison.
Job submission overview Marco Mambelli – August OSG Summer Workshop TTU - Lubbock, TX THE UNIVERSITY OF CHICAGO.
Condor Project Computer Sciences Department University of Wisconsin-Madison Condor-G: Condor and Grid Computing.
Batch Queue Prediction Warren Smith. 12/1/2009 TeraGrid Quarterly - Batch Queue Prediction2 Recommendation Scheduling Working Group recommends that TeraGrid.
Introduction to the Grid and the glideinWMS architecture Tuesday morning, 11:15am Igor Sfiligoi Leader of the OSG Glidein Factory Operations University.
Madison, Apr 2010Igor Sfiligoi1 Condor World 2010 Condor-G – A few lessons learned by Igor UCSD.
Honolulu - Oct 31st, 2007 Using Glideins to Maximize Scientific Output 1 IEEE NSS 2007 Making Science in the Grid World - Using Glideins to Maximize Scientific.
Workload Management Workpackage
SC’07 Demo Draft VGrADS Team June 2007.
Quick Architecture Overview INFN HTCondor Workshop Oct 2016
Operating a glideinWMS frontend by Igor Sfiligoi (UCSD)
High Availability in HTCondor
Joker: Getting the most out of the slurm scheduler
the challenge... ask not what your country can do for you
Building Grids with Condor
HTCondor Command Line Monitoring Tool
Making PowerShell Useful
Genre1: Condor Grid: CSECCR
GRID Workload Management System for CMS fall production
Condor-G: An Update.
Presentation transcript:

Interfacing SCMSWeb with Condor-G – A Joint PRAGMA-Condor Effort

sugree Hello, My name is Sugree from ThaiGrid

sugree I'm here to present our joint effort between Condor team and ThaiGrid team under PRAGMA

sugree Somsak from ThaiGrid → Develop the code Jaime from Condor → Condor installation & support Jysoo from KISTI → application test Cindy from SDSC → Coordination

sugree So you can submit Condor-G job to PRAGMA resources

sugree

Job script is exactly the same as Condor-G job script

sugree Condor-G will pick the (probably) best resources according to data from SCMSWeb

sugree Simple job script Executable = testcmd Universe = grid x509userproxy = /tmp/x509up_u533 grid_resource = $$(resource_name) # in case you want some specific cluster requirements = ( TARGET.name == "fsvc001.asc.hpcc.jp" ) output = test.out error = test.err Log = test-grid.log Queue # condor_submit condor_job_script

sugree ● Get the list of managed resources condor_status -startd

sugree ● Get the queue list condor_q

sugree For more information about job submission, please ask Jysoo, he is a lot better than me

sugree Now the interfacing module is working on rocks-153.sdsc.edu (same old Condor server)

sugree Job scheduling is quite slow. Should be more suitable for long-running job

sugree To get an account on the server, please contact Cindy

sugree It is possible to use this on your own server, while using data from PRAGMA-GOC.

sugree ● Future work – Paper submitted to IEEE eScience PRAGMA workshop – Find a better way to deal with error – Optimize Condor scheduling

sugree Any question?