Using a FreeBSD “cluster” to provide network services David Siebörger Systems Administrator I.T. Division, Rhodes University

Introduction We’ve set up a group of FreeBSD machines to provide network services on a redundant basis with load-sharing Not talking about a number-crunching, supercomputing cluster

What is FreeBSD? Free UNIX What do we do with it? –Almost* everything –DNS, DHCP, NTP, RADIUS, MySQL, WWW proxy, WWW server, Much of this could apply equally to Linux

elephant SMTP Spam Antivirus IMAP RT Webmail RADIUS Before porcupine Firewall hedgehog Firewall hippo DNS Proxy News Printing skink DHCP Proxy Backups squirrel Proxy News cache iguana Web apps lizard Web apps Monitoring RADIUS NTP terrapin DNS TFTP Syslog DHCP NTP Traffic accounting SMTP

Problems Too many different servers to manage Changing load patterns cause chaos

elephant SMTP Spam Antivirus IMAPRT The cluster design porcupine Firewall hedgehog Firewall hippo mowgli Backups DNS master Staging area squirreliguana Web apps lizard Web apps Monitoring terrapin RADIUS DNS DHCP Proxy Webmail NTP SMTP Antivirus RADIUS DNS DHCP Proxy Webmail NTP SMTP Antivirus RADIUS DNS DHCP Proxy Webmail NTP SMTP Antivirus Traffic accounting

The cluster design hipposquirrelterrapinporcupinehedgehog mowgli clusterfirewall Software and configuration copied from staging areas “jail” virtual servers running on mowgli

Load balancing strategies Handled differently for each service DHCP: it’s likely that all of the DHCP servers will answer all requests WWW proxy: the autoconfigure script is generated by PHP and lists the servers in random order SMTP: mail.ru.ac.za has multiple A records with short TTLs (round-robin DNS)

Failover strategies Handled differently for each service DHCP: at least one of the nodes will answer each request WWW proxy: the autoconfigure script lists multiple proxies. If the browser can’t connect to one, it picks another SMTP: do a DNS update to remove the A record for a server if it’s down

Synchronisation strategies In an ideal world, we’d use some sort of cluster filesystem or SAN rsync is a really simple alternative Copy everything from the staging area to each node, ignoring /var and /usr/ports It turns out we need to sync < 700MB and it’s quick There are other alternatives (rdist, unison)

Cluster addressing (IPv4) prefix: hippo a squirrel b terrapin f name.ru.ac.za x.dhcp.ru.ac.za x.mail.ru.ac.za x.cache.ru.ac.za x.radius.ru.ac.za x.ntp.ru.ac.za x.dns.ru.ac.za x.ldap.ru.ac.za

Cluster addressing (IPv6) prefix: 2002:548:1010:ff00hippo a squirrel b terrapin f name.ip6.ru.ac.za::1:1::1:2::1:6 x.dhcp.ip6.ru.ac.za::67:1::67:2::67:6 x.mail.ip6.ru.ac.za::25:1::25:2::25:6 x.cache.ip6.ru.ac.za::3128:1::3128:2::3128:6 x.radius.ip6.ru.ac.za::1812:1::1812:2::1812:6 x.ntp.ip6.ru.ac.za::123:1::123:2::123:6 x.dns.ip6.ru.ac.za::53:1::53:2::53:6 x.ldap.ip6.ru.ac.za::389:1::389:2::389:6

Advantages Scalability: it’s easy to add more nodes to to handle increased demand No wasted CPUs: all CPUs are available for every application Cheap hardware: single-processor 1U machines rather than SMP machines

Advantages Backups: we need only back up the staging area. Nothing on the nodes is irreplaceable (except perhaps log files) Redundancy: if one node fails, so what? Geographical dispersion: we can put nodes in switching centres around campus

Disadvantages Memory wastage: all applications are in memory on all nodes –But memory is cheap, and we can use standard memory in the 1U servers Disk wastage: we need n disks –But Serial ATA disks are really cheap Setup time: it’s taking us a while to get everything tidied-up