1. Jonathan Reed jdreed@mit.edu
AFS
Jonathan Reed

2. What is AFS? The Andrew File System, originally developed at CMU
For years, it was proprietary, sold by Transarc Corporation, and cost lots of money.
Transarc became IBM Pittsburgh Labs, and IBM released the source as OpenAFS.

3. Features of AFS
Location Transparency (you don’t need to know the address/hostname of the fileserver)
Caching
Secure Authentication
Scalability (designed to perform well with 200:1 client:server ratio)
Encryption (for OpenAFS)
Uniform namespace (path to a given file is the same regardless of client machine)

4. Components of AFS Basic OverSeer Service (bos) Cache Manager (afsd)
File Server (fs)
Kerberos Authentication Server (kas)
Kerberos Authentication DataBase (kadb)
Kernel module (libafs)
ProTection Server (pts)
PRotection DataBase (prdb)
VOlume Server (vos)
Volume Location DataBase (vldb)

5. Basic OverSeer Server
Monitors and controls AFS servers. bosserver runs on servers, bos client is available on all client machines.
Some things like status, require no authentication; other things (like reboot) do.

6. File Server
Runs on all AFS servers, and basically provides the same services provided by a local filesystem (ie: maintains hierarchy, delivers data, grants locks, creates links, checks permissions, etc)

7. Kerberos Authentication Server
Provides tokens to users, using Kerberos algorithms
Also provides authentication between clients and servers.
Not used at MIT - users are already authenticated by Kerberos for login
Not to be confused with the regular Kerberos server (kdc)

8. ProTection Server
AFS’ extension of the normal UNIX protection/permissions model
Provides seven access permissions (rlidwka) as opposed to UNIX’s three (rwx)
Uses an ACL instead of mode bits
Different permissions can be granted to several users, or several groups, instead of UNIX’s o/g/a
Can also specify permissions based on client machine’s IP
All info stored in PRotection DataBase (prdb)
Maintains AFS UID->username mappings so that the file server (which contains UIDs) can understand the token (which contains usernames)

9. VOlume Server
Allows for creation, deletion, and alteration of volumes, as well as tape archival. Also allows for replication.
Also deals with backups (OldFiles)
Volumes and Replication will be discussed in detail later

10. Volume Location Server
Maintains a list of volume locations in the Volume Location DataBase (vldb).
Provides information to the Cache Manager so it knows what File Server to talk to

11. Cache Manager Not a single process, like the others Initiated by afsd
Part of the kernel - the only part of AFS that runs on a client machine
Handles Rx events (Rx is AFS’ version of remote procedure calls (RPC). It’s what sunrpc is for NFS.
Caches Data.
Tracks the state of cached files via callbacks sent by File Server. To indicate that a file has changed, the File Server breaks a callback, and the Cache manager requests a new copy

12. How you get a file from AFS
The Cache Manager contacts the Volume Location Server server to find the File Server that contains a specific file. It translates the file request into an RPC (remote procedure call) to the File Server. The Cache Manager gets the file back, and it caches it before passing the data back to the requesting program.

13. Cells
A cell refers to a site running AFS (that is, a collection of servers and clients).
Cells are independently administered (ie: both the athena and sipb cells are at MIT, but are administered by different people)
Machines can only belong to one cell
Users belong to a cell in that they have an account there, but they can belong to many cells
Your local (or “home”) cell, is the one in which you initially authenticate, and is the one referred to by ThisCell (more later)
All other cells are “foreign” cells.

14. Volume
A volume is a container for a set of files. They can be any size, but must be smaller than a partition (by definition)
Volumes can be on one of many machines
Each volume corresponds to a directory in the file tree
Volumes are how resources are managed (each volume has a quota)
At MIT, and in most installations, volumes are named with a prefix that specifies what they contain. For example, user.paco is paco’s home directory. contrib.consult.readonly is the consult locker. project.outland is the outland locker in the sipb cell.
At MIT, volumes get renamed from user.foo to Xuser.foo when user foo is deactivated.

15. Replication
Read/write volumes can be replicated - a read-only clone of the volume is placed on different fileservers, so it’s available even if something bad happens.
volume.readonly indicates that it’s replicated
Most software volumes are replicated

16. Files in /usr/vice/etc
CellAlias - aliases (/afs/athena/ vs /afs/athena.mit.edu/)
SuidCells - “trusted” cells that we allow setuid programs from
ThisCell - the cell you’re in
CellServDB - maps file servers to cells - if you’re not listed in here, other cells can’t see you. Updated 3-4x per year.

17. AFS client startup
Magic is performed to determine the appropriate module to load (on Linux); module is loaded into kernel
afsd is started, and the cache is initialized.
encryption is negotiated (if possible and enabled)
Copy CellServDB, CellAlias, and SuidCells from /afs/athena.mit.edu/service/
Update local CellServDB if necessary.
Set permissions to allow setuid programs from the cells in SuidCells
Update cell aliases if necessary

18. AFS/NFS Translation
There are two AFS/NFS translators. Use them at your own risk. If it breaks, you get to keep both pieces. If you recommend it to a client, we’ll break you in two pieces. If you ask for support, you’ll be laughed at. Really, these are unsupported.
ni.mit.edu (aka afs2ftp.mit.edu) and atalanta.mit.edu
They also support anonymous ftp to obtain files in your Public directory, or any other “system:anyuser rl” directory.

19. Common Questions
Is AFS encrypted? Yes, on Solaris and Linux (OpenAFS) Regardless, you shouldn’t store sensitive information in plain text files.
Is there an AFS/NFS translator? Yes, but if you mention it to a client, both ops and I will kill you.
Do UNIX permissions work? Yes, but only the “user” or “owner” bits. You can set group/world bits, but AFS bits take priority.
Can I create hard links? Yes, but they have to stay within the same directory. (otherwise, which ACL is applied?) And it’s a bad idea anyway. Use symbolic ones.
How work? It is replaced by the system type (the output of fs sysname) transparently when a path is accessed.

20. AFS Command Suites
Command suites are “meta” commands. They do nothing by themselves, but take additional commands and arguments.
Most commands within the suites have aliases for convenience.
All command suites understand the “help” command, which lists the available commands within the suite.
bos - Controls bosserver
fs - Controls Cache Manager/File Server
pts - Interface to Protection Server
vos - Interface to Volume Server

21. bos You can’t do much without bits, but it’s useful for example.
bos status servername.mit.edu
bos listusers servername.mit.edu will tell you who the gods are.

22. fs
You’ll use this most frequently. You already know the setacl and listacl (sa and la, respectively) commands.
Other useful ones are
checks - check if file servers are up
lq - list quota and partition usage in both KB and percentage
quota - list quota usage in percentage
lsmount, rmmount, mkmount - control mount points, like OldFiles.
whereis, and whichcell - display the server and cell (respectively) of a given file

23. AFS Acls Specified for user or group.
Seven bits (Read, Write, List, Insert, Delete, locK, Admin)
Specified as the letter for each bit, or as one of four words (“read”, “write”, “all”, “none”).
fs sa dir {user | group} bits
fs la dir (dir defaults to current directory)
Groups are specifed as system:groupname (ie: system:athena-rcc)
Special groups:
system:anyuser - anyone
system:authuser - anyone with tokens for the cell
system:expunge - (specific to MIT - for expunging of deleted files)

24. pts
useful for examining AFS group information. In theory it should always sync with moira, but sometimes bad things happen.
pts mem system:group
pts exa user or system:group
pts exa outputs flags for control of pts commands on that user/group.
Flags are: s (examine), o (listowned), m (membership), a (adduser), r (removeuser).
Values are: lowercase letter (members of the group), uppercase letter (everyone), or “-” (system:administrators).
These are inherited from moira. Thus, if a list is visible, it gets a M flag.

25. vos
“examine” is likely to be the only one you’ll use. Examine takes a volume name. Forgot your volume name? fs lq will tell you, as will fs lsm, but fs lsm doesn’t take “.” as an argument
vos exa user.paco; vos exa sw.matlab
listvol tells you what volumes are on a given server. Not terribly useful.

26. Tokens Tokens authenticate you to AFS.
Based on AFS UID (not the same as UNIX UID)
Token manipulation:
tokens - display your tokens
unlog - kill your tokens
aklog [-force] - get your tokens, -force forces it to get new tokens even if you have some. (Necessary if group membership changes and you want access immediately)

27. Debugging The fstrace command provides cache manager debugging
fstrace sets cm -active
(Try to break something)
fstrace sets cm -inactive
fstrace dump

28. References www.openafs.org www.transarc.com
grand.central.org - good AFS FAQ
Arla: a free AFS client - started before OpenAFS