Download presentation
Presentation is loading. Please wait.
1
Condor-G: An Update
2
Outline What is Condor-G Past Present Future
3
What Is Condor-G Use Condor to run jobs on the Grid
Uses Globus Toolkit GRAM (submit a remote job) GASS (transfer job’s files) Two components Globus Universe GlideIn
4
Globus Universe Run a job on a Grid resource Features Disadvantages
Job management Fault tolerance Credential management Disadvantages No remote syscalls, checkpoint/migration, or dynamic resource selection
5
How It Works Condor-G Grid Resource Schedd LSF
6
600 Globus jobs How It Works Condor-G Grid Resource Schedd LSF
7
How It Works Condor-G Grid Resource Schedd LSF GridManager 600 Globus
jobs How It Works Condor-G Grid Resource Schedd LSF GridManager
8
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
600 Globus jobs How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
9
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
600 Globus jobs How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager User Job
10
GlideIn Create your own personal Condor pool from temporarily-acquired Grid resources Brings the full power of Condor to the Grid Run a Condor startd on a Grid resource Startd reports back to your machine and runs Vanilla and Standard Universe jobs
11
How It Works Condor-G Grid Resource Schedd LSF Collector 600 Condor
jobs How It Works Condor-G Grid Resource Schedd LSF Collector
12
How It Works Condor-G Grid Resource Schedd LSF Collector 600 Condor
jobs How It Works Condor-G Grid Resource Schedd glide-ins LSF Collector
13
How It Works Condor-G Grid Resource Schedd LSF GridManager Collector
jobs How It Works Condor-G Grid Resource Schedd glide-ins LSF GridManager Collector
14
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
jobs How It Works Condor-G Grid Resource JobManager Schedd glide-ins LSF GridManager Collector
15
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
jobs How It Works Condor-G Grid Resource JobManager Schedd glide-ins LSF GridManager Startd Collector
16
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
jobs How It Works Condor-G Grid Resource JobManager Schedd glide-ins LSF GridManager Startd Collector
17
How It Works Condor-G Grid Resource JobManager Schedd LSF GridManager
jobs How It Works Condor-G Grid Resource JobManager Schedd glide-ins LSF GridManager Startd Collector User Job
18
Globus Grid PBS LSF Condor Condor-G
19
600 Condor jobs Globus Grid PBS LSF Condor Condor-G
20
600 Condor jobs Globus Grid PBS LSF Condor Condor-G
21
600 Condor jobs Globus Grid PBS LSF Condor Condor-G glide-ins
22
600 Condor jobs Globus Grid PBS LSF Condor Condor-G glide-ins
23
600 Condor jobs Globus Grid PBS LSF Condor Condor-G glide-ins
24
600 Condor jobs Globus Grid PBS LSF Condor Condor-G glide-ins
25
Past GridManager daemon Globus GRAM 1.5
Runs Grid jobs using GRAM protocol Stages executable and standard I/O using GASS protocol Globus GRAM 1.5 We added fault-tolerance to the GRAM protocol Changes included in Globus Toolkit 2.0 release
26
Present Updated Condor-G to Globus Toolkit 2.0 Enhanced GridManager
GAHP
27
Enhanced GridManager Put problem jobs on hold No more stuck jobs
Increase concurrency with GAHP Almost ready
28
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
29
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
30
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
31
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
32
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
33
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
34
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
35
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
36
Single-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
37
Multi-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
38
Multi-Threaded Execution
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
39
Globus Application Helper Protocol (GAHP)
Condor is non-threaded Want to use multi-threaded libraries Increased concurrency Put libraries in external helper process Simple interface over pipes/sockets
40
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource Grid Resource
41
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
42
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
43
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
44
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
45
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
46
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
47
Multi-Threaded Execution with GAHP
Grid Resource GridManager Job 1 Grid Resource Job 2 Job 3 Job 4 Grid Resource GAHP Server GAHP Client Grid Resource
48
Future GRAM 1.6 Condor-G on Windows Condor-G Grid service
49
Globus GRAM 1.6 Working with Globus team to add additional features to GRAM protocol Credential refresh File staging Scheduler-specific options
50
Condor-G for Windows Condor GRAM and GASS APIs Condor-G
Windows implementation available GRAM and GASS APIs No C implementation for Windows (yet) Java implementation (Java CoG) Condor-G Windows version possible by writing GAHP server in Java
51
Condor-G Grid Service Reliable job submission service for higher-lever applications Open Grid Services Architecture (OGSA) SOAP, WSDL, WS-Inspection Implement Grid service interface for Condor-G (and Condor in general)
52
Thank You Condor-G demo on Wednesday Questions? 3351 CS Talk to me
Similar presentations
