1. Tutorial: Condor on Windows

2. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

3. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

4. Simple Job Submission

5. condor_store_cred
Contacts local SchedD and asks it to securely store a user’s password
Password is placed encrypted in a registry location
myp4sswd
y0urs
C:\>condor_store_cred add
Account:
Enter password:
Operation succeeded.

6. Job Execution: Submit Side
SchedD
Secure Password Store
Submit
myp4sswd
y0urs
Submit
Shadow

7. Job Execution: Execute Side
Starter
Jobs run using a Condor-specific account with minimal privileges.
Job
condor-reuse-slot1

8. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

9. Accessing Files on a Network Share
The condor-reuse-slotX accounts are local accounts without network privileges
To access files on a network share, the job needs to run using an account that has access to the shared network resources
Using a CredD, its possible to have jobs run with the credentials of the submitting user

10. The Windows CredD A centralized repository for user passwords CredD
myp4sswd
y0urs
“store password”
C:\>condor_store_cred add
Account:
Enter password:
Operation succeeded.
CredD
<password>

11. The Windows CredD SchedD Shadow
myp4sswd
“fetch password”
y0urs
<password>
Shadow
Submit machines can use the CredD to impersonate the user in the shadow

12. The Windows CredD Starter Job
“fetch password”
myp4sswd
y0urs
<password>
Job
Execute machines can use the CredD to run jobs as the desired user

13. Securing the CredD
NTSSPI can be used to authenticate to CredD and send the password encrypted over the network
myp4sswd
y0urs
C:\>condor_store_cred add
Account:
Enter password:
Operation succeeded.
CredD
“store password”

14. NTSSPI Authentication
Uses Windows built-in authentication capability
Used for user-to-daemon authentication
Always works when client and server are on the same machine
Can also be used over the network if the user account exists on both the client and server and the passwords match

15. Securing the CredD Starter Job
“fetch password”
myp4sswd
y0urs
Job
Condor normally runs as SYSTEM, and therefore can’t use NTSSPI

16. Securing the CredD Options for securing password fetch operations
Kerberos
SSL
Password

17. Password Authentication
Daemon-to-daemon communication is secured using a shared secret (the “pool password”)
Pool password must be distributed to all machines in the pool before enabling
Pool password is stored using the “-c” option to condor_store_cred
Password can be set remotely using “-n” option (make sure connection is authenticated!)

18. Setting the Pool Password
Enable CONFIG access from the machine that you will run condor_store_cred:
If setting the password remotely, ensure the user account that runs the condor_store_cred command exists on every machine and has a common password
Set the pool password on every machine:
HOSTALLOW_CONFIG = $(HOSTALLOW_ADMINISTRATOR)
C:\> condor_store_cred add –c –n node01.cs.wisc.edu

19. Configuring the CredD Host
Once pool password has been distributed, the CredD can be configured:
# Set security settings so that full security to the credd is required
CREDD.SEC_DEFAULT_AUTHENTICATION = REQUIRED
CREDD.SEC_DEFAULT_ENCRYPTION = REQUIRED
CREDD.SEC_DEFAULT_INTEGRITY = REQUIRED
CREDD.SEC_DEFAULT_NEGOTIATION = REQUIRED
# Require NTSSPI auth for password storing
CREDD.SEC_WRITE_AUTHENTICATION_METHODS = NTSSPI
# Require PASSWORD auth for password fetching
CREDD.SEC_DAEMON_AUTHENTICATION_METHODS = PASSWORD

20. Configuring the CredD Host
Use the condor_config.local.credd example file:
Include this file as one of the local configuration files:
C:\Condor> copy etc\condor_config.local.credd .\
LOCAL_CONFIG_FILE = $(LOCAL_DIR)/condor_config.local \
$(LOCAL_DIR)/condor_config.local.credd

21. Running Jobs as Submitting User
Submit and execute machines need to be configured to use the CredD:
CREDD_HOST = vault.cs.wisc.edu
STARTER_ALLOW_RUNAS_OWNER = True
SEC_CLIENT_AUTHENTICATION_METHODS = \ NTSSPI, PASSWORD
CREDD_CACHE_LOCALLY = True

22. Running Jobs as Submitting User
Example submit file:
universe = vanilla
executable = whoami.exe
log = whoami.log
output = whoami.out
run_as_owner = true
queue

23. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

24. Jobs with DLL Dependencies

25. Jobs with DLL Dependencies
Two ways to deal with DLL dependencies:
Ship the DLLs to the execute machine with the job
Make sure the job runs on an execute machines with the needed DLLs, and knows where to find them

26. Shipping DLLs with the Job
universe = vanilla
initial_dir = c:\cygwin
executable = sleep.exe
log = sleep.log
transfer_input_files = cygwin1.dll,cygintl-8.dll,cygiconv-2.dll
queue

27. Using DLLs on the Execute Side
Windows searches directories in the PATH environment variable for needed DLLs
C:\> set PATH=%PATH%;c:\cygwin\bin
universe = vanilla
executable = c:\cygwin\sleep.exe
log = sleep.log
getenv = true
queue

28. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

29. Checkpointing in Vanilla Universe
No Standard universe on Windows
Some jobs can do their own checkpointing
Condor will “warn” your job before eviction by sending the WM_CLOSE window message
For console applications, this translates to a CTRL_CLOSE_EVENT

30. Checkpointing in Vanilla Universe
Catching a “eviction notice” in a C-based console app:
Need to let Condor know that partial results are valuable in the submit file:
SetConsoleCtrlHandler(ckpt_function, TRUE)
when_to_transfer_output = ON_EXIT_OR_EVICT

31. Overview Simple job submission Accessing network shares
Handling DLL dependencies
Checkpointing in Vanilla universe
Submitting interpreted scripts

32. Submitting Scripts
Can just submit the interpreter as the executable and the script as an input file
executable = c:\perl\bin\perl.exe
arguments = script.pl args for script
transfer_executable = false

33. Submitting Scripts
If UNIX-style “#!” behavior is desired, use a USER_JOB_WRAPPER
Useful for using a single submit file that works on all platforms
executable = script.pl
arguments = args for script

34. USER_JOB_WRAPPER
# open the actual executable so we can look for interpreters #
if (open(EXECUTABLE, $ARGV[0])) {
# see if the first two bytes are #!
my $two_bytes;
if ((read(EXECUTABLE, $two_bytes, 2) == 2) && ($two_bytes eq "#!")) {
# look for the /usr/bin/perl interpreter
if (<EXECUTABLE> =~ /\/usr\/bin\/perl/) {
"c:\\cygwin\\bin\\perl.exe"); }

35. Questions?
Windows BOF:
Thursday, 11:00 – 12:00
Room 221