Presentation is loading. Please wait.

Presentation is loading. Please wait.

Condor-G Making Condor Grid Enabled

Published byCharles Bishop Modified over 5 years ago

Similar presentations

Presentation on theme: "Condor-G Making Condor Grid Enabled"— Presentation transcript:

1 Condor-G Making Condor Grid Enabled

2 Outline Why use Condor-G Globus Universe GlideIn Status & Future Work

3 What is Condor-G? Extensions to Condor to allow access to the Grid through Globus Two Parts Globus Universe GlideIn

4 Why Use Condor-G Condor Globus Condor-G
Designed to run jobs within a single administrative domain Globus Designed to run jobs across many administrative domains Condor-G Combine the strengths of both

5 Condor-G Helps Condor Users
Machines available to Condor users are limited Local Condor Pool Friendly Condor Pools (via Flocking) Through Globus, many more machines become available to run your jobs

6 Condor-G Helps Globus Users
Globus is primarily an infrastructure upon which to develop distributed applications Command-line tools are limited Some users don’t want to rewrite their applications to use Globus Condor-G provides them a powerful interface to the Grid to run their existing applications

7 Globus Universe Advantages of using Condor as a front-end to Globus
Full-featured queuing service Fault-tolerance Credential Management

8 Full-Featured Queue Persistent queue Many queue-manipulation tools
Set up job dependencies (DAGman) notification of events Log files

9 Fault-Tolerance Local Crash Remote Crash Queue state kept on disk
Condor Master restarts other daemons Remote Crash Condor will resubmit jobs Globus jobmanager enhanced to improve recoverability

10 Credential Management
Authentication in Globus is done with limited-lifetime X509 proxies Proxy may expire before jobs finish executing Condor can put jobs on hold and user to refresh proxy

11 How It Works Personal Condor Globus Resource Schedd LSF

12 How It Works Personal Condor Globus Resource Schedd LSF 600 Globus
jobs How It Works Personal Condor Globus Resource Schedd LSF

13 How It Works Personal Condor Globus Resource Schedd LSF GridManager
jobs How It Works Personal Condor Globus Resource Schedd LSF GridManager

14 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd LSF GridManager

15 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd LSF GridManager User Job

16 Globus Universe Disadvantages
No matchmaking or dynamic scheduling of jobs No job checkpoint or migration No remote system calls

17 Solution: GlideIn Use the Globus Universe to run the Condor daemons on Globus resources When the resources run these GlideIn jobs, they will join your Condor Pool Submit your jobs as Standard or Vanilla Universe jobs and they will be matched and run on the Globus resources

18 How It Works Personal Condor Globus Resource Schedd LSF Collector
jobs How It Works Personal Condor Globus Resource Schedd LSF Collector

19 How It Works Personal Condor Globus Resource Schedd LSF Collector
jobs How It Works Personal Condor Globus Resource Schedd glide-ins LSF Collector

20 How It Works Personal Condor Globus Resource Schedd LSF GridManager
jobs How It Works Personal Condor Globus Resource Schedd glide-ins LSF GridManager Collector

21 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd glide-ins LSF GridManager Collector

22 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd glide-ins LSF GridManager Startd Collector

23 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd glide-ins LSF GridManager Startd Collector

24 How It Works Personal Condor Globus Resource JobManager Schedd LSF
jobs How It Works Personal Condor Globus Resource JobManager Schedd glide-ins LSF GridManager Startd Collector User Job

25 GlideIn Concerns What if a Globus resource kills my GlideIn?
That resource will disappear from your pool and you jobs will be rescheduled on other machines What if all my jobs are done before a GlideIn runs? If the glided-in Condor daemons are not matched with a job in 10 minutes, they terminate

26 LSF PBS Globus Grid Condor personal Condor your workstation Group

27 LSF PBS Globus Grid Condor your workstation personal Condor Group
jobs Globus Grid PBS LSF Condor your workstation personal Condor Group Condor

28 LSF PBS Globus Grid Condor Group Condor personal Condor your
jobs Globus Grid PBS LSF Condor Group Condor personal Condor your workstation

29 LSF PBS Globus Grid Condor Group Condor your workstation personal
jobs Globus Grid PBS LSF Condor Group Condor your workstation personal Condor glide-ins

30 LSF PBS Globus Grid Condor Group Condor your workstation personal
jobs Globus Grid PBS LSF Condor Group Condor your workstation personal Condor glide-ins

31 LSF PBS Globus Grid Condor Group Condor your workstation personal
jobs Globus Grid PBS LSF Condor Group Condor your workstation personal Condor glide-ins

32 LSF PBS Globus Grid Condor Group Condor your workstation personal
jobs Globus Grid PBS LSF Condor Group Condor your workstation personal Condor glide-ins

33 LSF PBS Globus Grid Condor Group Condor your workstation personal
jobs Globus Grid PBS LSF Condor Group Condor your workstation personal Condor glide-ins

34 Current Status First version of GridManager ready
Runs jobs using Globus GRAM Stages executable and standard I/O using Globus GASS Jobmanager changes will be folded into a future release of Globus Credential management in progress

35 Future Work GridManager Automatic GlideIn
Stage user jobs’ data files Automatic GlideIn Condor creates GlideIn jobs when more resources are needed Matchmaking in Globus Universe Use Globus GRIS to create ClassAds for Globus resources and match them to job ClassAds

36 Questions and Thank You!

Download ppt "Condor-G Making Condor Grid Enabled"

Similar presentations

Jaime Frey, Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison OGF.

Jaime Frey, Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison OGF.
Jaime Frey Computer Sciences Department University of Wisconsin-Madison OGF 19 Condor Software Forum Routing.

Jaime Frey Computer Sciences Department University of Wisconsin-Madison OGF 19 Condor Software Forum Routing.
Condor Project Computer Sciences Department University of Wisconsin-Madison Eager, Lazy, and Just-in-Time.

Condor Project Computer Sciences Department University of Wisconsin-Madison Eager, Lazy, and Just-in-Time.
Dan Bradley Computer Sciences Department University of Wisconsin-Madison Schedd On The Side.

Dan Bradley Computer Sciences Department University of Wisconsin-Madison Schedd On The Side.
CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.

CERN LCG Overview & Scaling challenges David Smith For LCG Deployment Group CERN HEPiX 2003, Vancouver.
Part 7: CondorG A: Condor-G B: Laboratory: CondorG.

Part 7: CondorG A: Condor-G B: Laboratory: CondorG.
June 21-25, 2004Lecture2: Grid Job Management1 Lecture 3 Grid Resources and Job Management Jaime Frey Condor Project, University of Wisconsin-Madison

June 21-25, 2004Lecture2: Grid Job Management1 Lecture 3 Grid Resources and Job Management Jaime Frey Condor Project, University of Wisconsin-Madison
1 Concepts of Condor and Condor-G Guy Warner. 2 Harvesting CPU time Teaching labs. + Researchers Often-idle processors!! Analyses constrained by CPU time!

1 Concepts of Condor and Condor-G Guy Warner. 2 Harvesting CPU time Teaching labs. + Researchers Often-idle processors!! Analyses constrained by CPU time!
Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.

Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.
A Computation Management Agent for Multi-Institutional Grids

A Computation Management Agent for Multi-Institutional Grids
Condor and GridShell How to Execute 1 Million Jobs on the Teragrid Jeffrey P. Gardner - PSC Edward Walker - TACC Miron Livney - U. Wisconsin Todd Tannenbaum.

Condor and GridShell How to Execute 1 Million Jobs on the Teragrid Jeffrey P. Gardner - PSC Edward Walker - TACC Miron Livney - U. Wisconsin Todd Tannenbaum.
Jaime Frey Computer Sciences Department University of Wisconsin-Madison Condor-G: A Case in Distributed.

Jaime Frey Computer Sciences Department University of Wisconsin-Madison Condor-G: A Case in Distributed.
GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.

GRID workload management system and CMS fall production Massimo Sgaravatto INFN Padova.
GRID Workload Management System Massimo Sgaravatto INFN Padova.

GRID Workload Management System Massimo Sgaravatto INFN Padova.
Workload Management Massimo Sgaravatto INFN Padova.

Workload Management Massimo Sgaravatto INFN Padova.
Alain Roy Computer Sciences Department University of Wisconsin-Madison 25-June-2002 Using Condor on the Grid.

Alain Roy Computer Sciences Department University of Wisconsin-Madison 25-June-2002 Using Condor on the Grid.
First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova

First steps implementing a High Throughput workload management system Massimo Sgaravatto INFN Padova
Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.

Evaluation of the Globus GRAM Service Massimo Sgaravatto INFN Padova.
Jaeyoung Yoon Computer Sciences Department University of Wisconsin-Madison Virtual Machines in Condor.

Jaeyoung Yoon Computer Sciences Department University of Wisconsin-Madison Virtual Machines in Condor.
Cheap cycles from the desktop to the dedicated cluster: combining opportunistic and dedicated scheduling with Condor Derek Wright Computer Sciences Department.

Cheap cycles from the desktop to the dedicated cluster: combining opportunistic and dedicated scheduling with Condor Derek Wright Computer Sciences Department.

Similar presentations

Ads by Google