1. Todd Tannenbaum Condor Team http://www.cs.wisc.edu/condor GCB Tutorial OGF 2007

2. www.cs.wisc.edu/condor What is GCB? GCB is the Generic Connection Broker Included in Condor 6.7.13 (Nov 2005) and later Linux-only It solves the firewall traversal problem So what is the firewall traversal problem?

3. www.cs.wisc.edu/condor A Simple Condor Pool Matchmaker Executor Submitter Communication is initiated in two directions Note: This is a subset of communication in Condor

4. www.cs.wisc.edu/condor What If There Is A Firewall? Firewalls usually block incoming traffic on most ports Incoming depends on your perspective: Organizations have firewalls to protect from computers outside the organization Individual computers have firewalls to protect from other computers

5. www.cs.wisc.edu/condor A Condor Pool With Firewall Matchmaker Executor Submitter X X

6. www.cs.wisc.edu/condor How Can You Traverse Firewalls? Punch a hole Configure firewall to allow traffic on a certain range of ports to come through Tell Condor to restrict itself to use only this range Bummer: Condor can use many ports Bummer: Punching holes makes people nervous

7. www.cs.wisc.edu/condor How Can You Traverse Firewalls? Use Condor-C Matchmaker Executor Submitter Re-Submitter Put host on network edge Open a couple of ports for it Delegate jobs to this host

8. www.cs.wisc.edu/condor How Can You Traverse Firewalls? Change Condor to always use outgoing traffic What if there are two firewalls or private networks? Which direction is outgoing? GCB automates this solution It knows which direction is outgoing It can proxy if there are two firewalls

9. www.cs.wisc.edu/condor GCB: Contacting Executor (One Possible Scenario) Matchmaker Executor Submitter GCB 4 1 1. Executor registers with GCB (Permanent TCP connection) 2. Executor advertises to matchmaker (GCB IP address) 2 3. After match, submitter contacts executor, via GCB 3 4. GCB tell executor to open connection 5. Executor opens connection to submitter 5

10. www.cs.wisc.edu/condor GCB (Acting as Proxy) Matchmaker Executor Submitter 1. Assume 1 port open for matchmaker. (Can avoid…) 2. Executor advertises with GCB (permanent connection) 3. Executor advertises to matchmaker (GCB IP address) 4. After match, submitter contacts executor, via GCB 5. Communication flows through GCB, using both connections 1 5 GCB 2 3 4

11. www.cs.wisc.edu/condor GCB Advantages Good connectivity Works with multiple private networks Works with network address translation Dont need to punch holes in firewall GCB does not need to be run as root No changes to firewall configuration

12. www.cs.wisc.edu/condor GCB Disadvantages GCB is a point of failure All communications through GCB, so if GCB fails… Computers behind a firewall share an IP address (of GCB) Makes host-based security difficult Doesnt work with Kerberos security Can slow down network performance Scalability issues A single GCB server is limited by number of ports available on computer Complex to configure and debug

13. www.cs.wisc.edu/condor Now for the Nitty Gritty…

14. www.cs.wisc.edu/condor Setting Up GCB 1. Install GCB 2. Configure GCB 3. Configure Condor to use GCB

15. www.cs.wisc.edu/condor Install GCB GCB comes with Condor GCB has two programs gcb_broker : The big brains of GCB gcb_relay_server : proxy for private net to private net communication GCB was written independently of Condor Cant read condor_config directly So create environment in condor_config GCB reads from environment

16. www.cs.wisc.edu/condor Install GCB GCB should be on computer with no other services GCB can use lots of ports, so avoid port competition with other programs Using GCB can slow down communication, so keeping GCB on its own computer helps speed GCB needs to be on edge of network On public network and private network At least one GCB per private network

17. www.cs.wisc.edu/condor Configure GCB To run from condor_master: # Specify that you only want the master # and the broker running DAEMON_LIST = MASTER, GCB_BROKER # Define the path to the broker binary # for the master to spawn GCB_BROKER=$(RELEASE_DIR)/libexec/gcb_broker

18. www.cs.wisc.edu/condor Configure GCB GCB expects configuration in environment. Sample: GCB_BROKER_ENVIRONMENT = # Provide the full path to the gcb_relay_server GCB_BROKER_ENVIRONMENT = GCB_RELAY_SERVER=$(GCB_RELAY) # Tell GCB to write all log files into the Condor log # directory GCB_BROKER_ENVIRONMENT=(GCB_BROKER_ENVIRONMENT);GCB_LOG_DIR=$(LOG) # Tell GCB it can connect to private network GCB_BROKER_ENVIRONMENT=$(GCB_BROKER_ENVIRONMENT);GCB_ACTIVE_TO_CLIENT=yes # Set public IP address for GCB broker GCB_BROKER_ARGS = -i 123.123.123.123 # Provide the full path to the gcb_relay_server GCB_BROKER_ENV = GCB_RELAY_SERVER=$(GCB_RELAY) # Tell GCB to write all log files into the # Condor log directory GCB_BROKER_ENV=$(GCB_BROKER_ENV);GCB_LOG_DIR=$(LOG) # Set public IP address for GCB broker GCB_BROKER_ARGS = -i 123.123.123.123 Note: more configuration options are available. See manual for details # Tell GCB it can connect to private network GCB_BROKER_ENV = $(GCB_BROKER_ENV);GCB_ACTIVE_TO_CLIENT=yes

19. www.cs.wisc.edu/condor Configure Condor to Use GCB In condor_config: Turn on GCB: NET_REMAP_ENABLE = true NET_REMAP_SERVICE = GCB # Point to GCB NET_REMAP_INAGENT = 123.123.123.123 # Routing Table NET_REMAP_ROUTE = /full/path/gcbroutes

20. www.cs.wisc.edu/condor Set Up Routing Table Private Network 192.168.2.* Public Network 123.123.123.* GCB Broker 123.123.123.123 Routing Table 123.123.123.123/32 GCB */0 direct

21. www.cs.wisc.edu/condor Set Up Routing Table Private Network 192.168.2.* Public Network 123.123.123.* GCB Broker 123.123.123.65 Routing Table 123.123.123.65/32 GCB 123.123.123.66/32 GCB */0 direct Private Network 192.168.2.* GCB Broker 123.123.123.66

22. www.cs.wisc.edu/condor Security Implications Hosts in private network look like they share a single IP Address (the address of the GCB broker) If you use host-based security, you cant distinguish hosts in the private network GCB does not authenticate who it is providing its proxy service for.

23. www.cs.wisc.edu/condor More Information Section 3.8 of the Condor manual Networking http://www.cs.wisc.edu/~sschang/firewall/gcb Thank You!!!