Condor and Grids.

Slides:



Advertisements
Similar presentations
Jaime Frey, Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison OGF.
Advertisements

Jaime Frey Computer Sciences Department University of Wisconsin-Madison OGF 19 Condor Software Forum Routing.
GridSAM Overview Grid Job S ubmission A nd M onitoring Service What is GridSAM? Funded by the OMII Managed Programme (Started in Sept, 04) Client Perspective.
Community Grids Lab1 CICC Project Meeting VOTable Developed VotableToSpreadsheet Service which accepts VOTable file location as an input, converts to Excel.
Dan Bradley Computer Sciences Department University of Wisconsin-Madison Schedd On The Side.
Part 7: CondorG A: Condor-G B: Laboratory: CondorG.
Security Daniel Mallmann MWSG meeting Amsterdam December 2005.
Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.
Condor DAGMan Warren Smith. 12/11/2009 TeraGrid Science Gateways Telecon2 Basics Condor provides workflow support with DAGMan Directed Acyclic Graph –Each.
Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
Grid Programming Environment (GPE) Grid Summer School, July 28, 2004 Ralf Ratering Intel - Parallel and Distributed Solutions Division (PDSD)
SSL Man in the Middle Proxy Srinivas Inguva Dan Boneh Ian Baker Stanford University.
First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova
Alain Roy Computer Sciences Department University of Wisconsin-Madison An Introduction To Condor International.
High Throughput Parallel Computing (HTPC) Dan Fraser, UChicago Greg Thain, Uwisc.
- 1 - Grid Programming Environment (GPE) Ralf Ratering Intel Parallel and Distributed Solutions Division (PDSD)
Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison What’s New in Condor.
Ashok Agarwal 1 BaBar MC Production on the Canadian Grid using a Web Services Approach Ashok Agarwal, Ron Desmarais, Ian Gable, Sergey Popov, Sydney Schaffer,
Progress Report Barnett Chiu Glidein Code Updates and Tests (1) Major modifications to condor_glidein code are as follows: 1. Command Options:
GRAM: Software Provider Forum Stuart Martin Computational Institute, University of Chicago & Argonne National Lab TeraGrid 2007 Madison, WI.
OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.
GRAM5 - A sustainable, scalable, reliable GRAM service Stuart Martin - UC/ANL.
1 Overview of the Application Hosting Environment Stefan Zasada University College London.
3-2.1 Topics Grid Computing Meta-schedulers –Condor-G –Gridway Distributed Resource Management Application (DRMAA) © 2010 B. Wilkinson/Clayton Ferner.
CSF4 Meta-Scheduler Name: Zhaohui Ding, Xiaohui Wei
Condor Project Computer Sciences Department University of Wisconsin-Madison Condor-G Operations.
Grid job submission using HTCondor Andrew Lahiff.
Condor: High-throughput Computing From Clusters to Grid Computing P. Kacsuk – M. Livny MTA SYTAKI – Univ. of Wisconsin-Madison
Mar 28, 20071/18 The OSG Resource Selection Service (ReSS) Gabriele Garzoglio OSG Resource Selection Service (ReSS) Don Petravick for Gabriele Garzoglio.
Report from USA Massimo Sgaravatto INFN Padova. Introduction Workload management system for productions Monte Carlo productions, data reconstructions.
Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison Condor RoadMap.
Condor-G A Quick Introduction Alan De Smet Condor Project University of Wisconsin - Madison.
Grid Security: Authentication Most Grids rely on a Public Key Infrastructure system for issuing credentials. Users are issued long term public and private.
July 11-15, 2005Lecture3: Grid Job Management1 Grid Compute Resources and Job Management.
EGEE-II INFSO-RI Enabling Grids for E-sciencE The GILDA training infrastructure.
Review of Condor,SGE,LSF,PBS
Condor Project Computer Sciences Department University of Wisconsin-Madison Grids and Condor Barcelona,
Grid, Web services and Taverna Machiel Jansen Richard Holland.
Pilot Factory using Schedd Glidein Barnett Chiu BNL
Jaime Frey Computer Sciences Department University of Wisconsin-Madison What’s New in Condor-G.
Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.
The OxGrid Resource Broker David Wallom. Overview OxGrid Resource Broking Why build our own Job Submission and other tools Future developments.
HTCondor’s Grid Universe Jaime Frey Center for High Throughput Computing Department of Computer Sciences University of Wisconsin-Madison.
First evaluation of the Globus GRAM service Massimo Sgaravatto INFN Padova.
Condor Project Computer Sciences Department University of Wisconsin-Madison Condor-G: Condor and Grid Computing.
HTCondor-CE. 2 The Open Science Grid OSG is a consortium of software, service and resource providers and researchers, from universities, national laboratories.
A GOS Interoperate Interface's Design & Implementation GOS Adapter For JSAGA Meng You BUAA.
UCS D OSG Summer School 2011 Intro to DHTC OSG Summer School An introduction to Distributed High-Throughput Computing with emphasis on Grid computing.
Madison, Apr 2010Igor Sfiligoi1 Condor World 2010 Condor-G – A few lessons learned by Igor UCSD.
Workload Management Workpackage
Quick Architecture Overview INFN HTCondor Workshop Oct 2016
Dynamic Deployment of VO Specific Condor Scheduler using GT4
Operating a glideinWMS frontend by Igor Sfiligoi (UCSD)
Peter Kacsuk – Sipos Gergely MTA SZTAKI
Outline Expand via Flocking Grid Universe in HTCondor ("Condor-G")
OMII evaluation: Preliminary results Current status of T6
Accounting in HTCondor
Accounting, Group Quotas, and User Priorities
Bridging Unicore and Condor
Using Condor An Introduction Condor Week 2006
Standard Portlet Architecture Supports Reusable Components
Genre1: Condor Grid: CSECCR
The Condor JobRouter.
OGCE Portal Software for Big Red and the TeraGrid
GRID Workload Management System for CMS fall production
OGCE Architecture: Portlets and Services for Science Gateways
Condor-G Making Condor Grid Enabled
JRA 1 Progress Report ETICS 2 All-Hands Meeting
Condor-G: An Update.
Presentation transcript:

Condor and Grids

Potential Topics Supported grid types Matchmaking GlideIn Credential handling Scalability Firewalls Monitoring & validation Troubleshooting

Potential Topics Supported grid types Matchmaking GlideIn Credential handling Scalability Firewalls Monitoring & validation Troubleshooting

Pre-WS GRAM Submit file grid_resource = gt2 \ foo.edu/jobmanager-pbs globus_rsl = (queue=long)\ (condor_submit=(universe java))

OGSA GRAM Submit file grid_resource = gt3 http://foo.edu/\ ogsa/services/base/gram/\ PBSManagedJobFactoryService globus_rsl = (queue=long)\ (condor_submit=(universe java)) Museum mode

WS GRAM Submit file grid_resource = gt4 foo.edu PBS globus_xml = <queue>long</queue>

NorduGrid Submit file grid_resource = nordugrid foo.edu nordugrid_rsl = (queue=long)

Unicore Submit file grid_resource = unicore usite.org vsite keystore_file = keystore keystore_passphrase_file = keystore.pw keystore_alias = my cert

Condor Submit file grid_resource = condor schedd.foo.edu \ cm.foo.edu remote_universe = java

PBS Submit file grid_resource = pbs

LSF Submit file grid_resource = lsf

Matchmaking Job Ad grid_resource = $$(ResourceName) requirements = TARGET.ResourceName \ =!= Undefined rank = random()

Matchmaking Machine Ad MyType = “Machine” TargetType = “Job” Name = “foo.edu” Machine = “foo.edu” ResourceName = “gt4 foo.edu PBS” UpdateSequenceNumber = 4 Requirements = TARGET.JobUniverse == 9 && \ CurMatches < 10 CurMatches = 0 Rank = 0.0 CurrentRank = 0.0 WantAdRevaluate = True

Matchmaking Advertising a resource condor_advertise UPDATE_STARTD_AD \ ad-file Call periodically Use unix time for UpdateSequenceNumber

MyProxy Submit file MyProxyHost = foo.edu:12345 MyProxyServerDN = /DC=org/DC=doegrids… MyProxyCredentialName = proxy_file MyProxyRefreshThreshold = 240 #mins MyProxyNewProxyLifetime = 12 #hrs MyProxyPassword = password Or give password on command line condor_submit -p password submit.desc