1. OpenAFS in a multihomed universe Jeffrey Altman Derrick Brashear

2. Topics This is really 2 discussions: Configuring and using multi-homed clients. Configuring and using multi-homed servers. It’s important to understand how we got here. Some issues are still “on the table”.

3. Background When AFS development began, multi- homed hosts were uncommon. Assumptions made early did not accommodate multi-homed hosts. As commercial (Transarc) customers began to require multi-homed support, changes were made over time to parts of AFS. Some offered better functionality than others. Full integration wasn’t always possible.

4. Architecture (abridged) VLserver Used to discover volume locations. Uses ubik distributed database technology to provide current copies of data, provided a fully-replicated service. Fileserver Registers its address with the VLserver. Used to provide data to clients. Must track clients to provide callback notifications. Client Uses VLserver to discover data location, then fetch it from fileservers on behalf of users.

5. Issues: VLserver How do clients find the VLserver in the first place? Hardcoded address in CellServDB. IBM never solved this; you had to maintain copies of the file if you needed to address some or all database servers by different addresses on different groups of clients.

6. Solutions: VLserver AFSDB Support for AFSDB DNS records added by OpenAFS volunteers. Use of split-horizon DNS allows different information to be distributed to different groups of clients, but still be managed in few places.

7. Issues: fileserver Original VL RPCs returned a location which was server and partition. Server meant “one IP address”. If you had a different address space than the one in which the VLserver was providing answers, you were sad.

8. Solutions: fileserver UUID Fileservers generate a “universally-unique identifier” which can be used in place of a single address. Included in the data structure with the actual UUID number is a list of addresses the fileserver registered with the VLserver. Revised VL RPCs use UUID instead of address; New clients and servers use the new RPCs when available.

9. Issues: client The client itself can have more than one address. Fileserver tracking of client by IP address only. A fixed client port of 7001 was assumed in some cases. Worse, the client can have an alternate port!

10. Solutions: client Clients also get a UUID. Fileserver callback interfaces changed to support UUIDs rather than addresses. Port assumptions removed.

11. 11 Complications It’s not that simple. NAT A client can be behind a NAT and not know its real address, precluding it from telling the server its actual address. A server can be behind a NAT! You may not want to advertise all of your addresses! 11

12. 12 Enter NetInfo and NetRestrict In order to only publish some addresses, or mask some addresses, a new interface was created, for both clients and servers. NetInfo is a list of addresses to publish. NetRestrict is a list of addresses to mask. Typically you use one or the other.

13. 13 NetInfo and NAT Remember servers can also be behind NAT! NetInfo supports fake addresses. The only gotcha here is that you need to know the “other” address of your server. If it changes, you need to update NetInfo and restart. For clients, it matters less. 13

14. Client addresses Who can you trust? Clients behind a NAT will not know their “real” external address. Advertised internal addresses may be non- routable. Fileserver updated to use the calling address and not use addresses advertised to it without first verifying they are the same host.

15. Forget your problems? Just because a host has multiple addresses doesn’t mean you want to use them all. Rx RPC library built on a single socket model. Allow socket to be bound to exactly one address. (“rxbind”) Requires list of addresses as modified by NetInfo and/or NetRestrict to be whittled to exactly one address to work.

16. Or at least make them manageable Clients support the notion of server preferences. Server preferences are by address, not UUID! Preference can be given to a particular address for a server. Sadly, the defaults are based on classful networking, which is not as useful as it could be.

17. Except when you can’t Ubik databases will have one primary address. True multi-homing is not supported in the internal ubik interfaces. Related, special powers are assigned always to the member of a ubik service pool with the lowest IP address.

18. What’s still needed Truly multi-homed Ubik. A way to change what addresses a fileserver is advertising, on the fly. A way to discover what addresses those should be other than manual configuration. Server preference support in clients which is not based on classful networks. Multi-socket Rx. Or maybe just finish Rx TCP and IPv6 support.

19. OpenAFS in a multi- homed universe Questions?