1. HTCondor Networking Concepts

2. Disclaimers Not about configuration macros
Not about host or daemon lookups
Not about HTCondor internals
Hopefully this is the last time I say those two words. [“configuration macros”]

3. Asking the Right Questions
There will be a quiz at the end
Start by reviewing fairy-tale networking
… then add IPv6
… then add schedd firewalls
… then add startd firewalls
End by passing the quiz (open-manual)

4. Fairy-tale Networking
Single network protocol
All addresses publically routable
No firewalls
Fewer than ~25k simultaneous running jobs

5. Working in a Fairy Tale negotiator collector schedd startd shadow*
starter*
* One shadow, starter per running job

6. IPv6
negotiator
collector
schedd
startd
shadow
starter
IPv4
IPv6

7. IPv6 + IPv4 negotiator collector schedd startd shadow starter startd

8. Shared Port Problem: Firewall Problem: only ~60k TCP ports
Admin willing to open only one port
Problem: only ~60k TCP ports
Need one per shadow
Shared Port Service
Listens on single port for incoming connections
Hands each connection to intended recipient

9. Shared Port
Internet
Fire wall
schedd
startd
shared_port
starter

10. Firewalled Submit Node
negotiator
collector
schedd
Wall
startd
shared port
starter
shadow
Fire

11. TCP Forwarding Host Problem: Private network with NAT
Traverse firewall via port forwarding
Allocate a public IP address
Connections to public address forwarded by NAT to machine on private network
Common in the Cloud

12. Condor Connection Broker
Problem: Private network with NAT
Or firewall with no opening for HTCondor
Traverse firewall by reversing connection
Client sends connection request via broker
Server initiates TCP connection to client
Only bypasses one firewall
Client and broker (CCB server) must have publically routable addresses

13. CCB: Condor Connection Broker
Internet
Outbound firewall
schedd
startd
schedd

14. NATd Execute Nodes negotiator collector/CCB schedd shared port startd
Wall
NAT
shared port
Fire
startd
shadow
starter

15. Port Usage (Digression)
Shadow for each running job
In fairy-tale setup
Each shadow uses two ports
Limit of ~25k running jobs
With shared port and CCB
Shadow use no ports
No network limit on number of running jobs

16. Quiz
Why do schedds and central managers need to be mixed-mode in a pool split between IPv4 and IPv6 nodes?
Why use CCB on execute nodes?
Why use both CCB and shared port?
If both the schedd and the execute nodes are NATd, what do you do?
4 -- fewer schedds; CCB’d schedds may not be able to flock; for NAT use port forwarding and set TCP_FORWARDING_HOST on schedd. HTCondor solutions may scale better than VPN (CCB does not actually forward); if schedd & execute nodes NATd on same network, can use PrivNet and PrivName.

17. Question 1
Why do schedds and central managers need to be mixed-mode in a pool split between IPv4 and IPv6 nodes?
They need to be able to talk to all execute nodes

18. Question 2 Why use CCB on execute nodes (and not submit nodes)?
Easier to make submit nodes publically accessible (fewer of them)

19. Question 3 Why use both CCB and shared port?
Can’t use CCB for both schedd and startd
No ports used for shadow, so no limit on number of running jobs

20. Question 4
If both the schedd and the execute nodes are NATd, what do you do?
If same NAT, no problem
TCP Forwarding Host for schedd

21. Congratulations!
HTCondor Administrator
Networking