Basic Grid Projects – Condor (Part I) Sathish Vadhiyar Sources/Credits: Condor Project web pages

Condor Motivation Most of the cycles (70%) of workstation pools are underutilized High throughput computing – Large amounts of processing capacity over long periods of time In contrast to High Performance Computing Support system with distributed ownerships Owners specify access policies

Condor Features Specialized workload management system Provides a job queueing mechanism, scheduling policy, resource monitoring, and resource management Can effectively harness wasted CPU power from otherwise idle desktop workstations Can checkpoint and migrate a job to a different machine

Condor Architecture – daemons / processes master Startd Represents a machine to the Condor pool Implement owner’s access control policies Starts, stops, suspends jobs Runs on executing machines Starter Spawned by startd for a job Coordinates with the job

Condor Architecture - daemons Schedd Represents jobs to the condor pool Maintain persistent queues of user’s requests Runs on submit machines Shadow Similar to starter functionality, but runs on submit machine Job specific Manager Collector Collects machine and resource information from all other daemons Answers queries Negotiator Retrieves information from collector Does match making

Job Submission Steps

Idle capacity utilization When owner returns, Condor checkpoints and migrates jobs

classads Language used in Condor For describing jobs, workstations, and other resources Mapping from attribute names to expressions Used by condor central manager to decide on job scheduling

Steps Step 1 – entities express their characteristics through classads Also expresses constraints for other entities to follow Also specifies ranks (preferences) to break ties in case there are more than one entities following the constraints. Step 2 – matchmaker matches different classads Step 3 – match maker notifies matched entities Step 4 – matched entities establish allocation

ClassAds

More on ClassAds Resource owners and customers can dynamically define own models – suitable for distributed setting Matching and claiming as 2 distinct operations 5 components of matchmaking protocol classAd specification advertising protocol matchmaking algorithm matchmaking protocol claiming protocol Constraints, i.e. queries, may be expressed as attributes of classAd classAd definition – mapping from attribute names to expressions

Examples

Examples