AFS Per-File ACLs Marc Dionne TechnoConseil

Outline Introduction History Issues -Protocol and semantics Issue - Implementation Issues - Compatibility Current status What's next

Introduction AFS user/administrator/developer for 15 years Mainly involved in development over the past few years –Linux client kernel updates, bugs, some improvements –Code cleanup efforts Looking for a more substantial and challenging contribution

Introduction AFS only allows ACLs to be set at the directory level –All files share this ACL Disadvantage compared to other local or network filesystems Users new to AFS expect ACLs to work on files Need workarounds for common situations: dot files,.mailrc, etc – Some files need to be in a common location, but need different rights Some demand for this feature in the community

History Had questions about the feasibility – in particular the impact on the client side Coded an initial prototype – june 2009 – over a weekend –Simplest implementation possible New special file for file ACLs, parallels small vnode index – 1 slot per file Existing ACL RPCs Encouraging - surprisingly stable and functional –Very few changes needed on the Unix client... but some issues lurking Discussions on afs3-std and other venues Several revisions since then

Issues - Protocol and semantics Current AFS protocol does not specify ACL operations on files –Requires new RPCs: fetchACL and storeACL Inheritance semantics need to be defined –Are ACLs inherited, when, how Inherit until set Inherit always –How does an ACL change on a directory affect files Aim for least surprise for current AFS users Behaviour of client tools –fs setacl, listacl –vos move, restore, etc.

Issues - Compatibility General –All combinations of current and new clients and servers should interoperate reasonably –OK to restrict new functionality – limit access to files that have an ACL with broader access than the parent –But not OK to expose files that file ACLs should make unreadable Servers and clients need to be aware of the other side’s status –Use client and file server capability bits in OpenAFS –Capabilities exchange for Unix client recently merged

Issues - Compatibility Current clients can leak cached data –They assume directory rights apply to all files, but rights may now vary per file –Scenario Users A and B can read directory D File F has a file ACL that allows A to read, but not B A reads F, data is brought into the cache B tries to read F, cache manager assumes rights to D apply, and happily returns data –Possible solution: artificially restrict rights Fiddle with the returned user rights on the server side, or the file mode bits Compute most restrictive rights for that user for all files within a directory – return the same rights for all files

Issues - Compatibility Windows –Tests showed the Windows client reacted badly with files more restricted than the parent directory Mainly lengthy hangs in explorer –Commit 9e8ae43b introduced a registry key to correct this behaviour Should be activated based on server capabilities Same solution should apply – return most restrictive rights in the directory Volume moves and restores –Prevent data (ACLs) loss while moving volumes to an older server –Allow forced moves

Issues - Implementation Changes are required to the existing on-disk structures On-disk Vnode structure is full –RXOSD already repurposes some elements (vnode “magic”) –Really need a pointer in the Vnode – alternatives are much more complex (hashing, etc.) –Current scheme requires a power of 2 size Small vnode size would have to double May be a concern for sites with large numbers of files Volume header is nearly full –RXOSD repurposes an existing file pointer New volume data (file ACLs) need to be preserved across volume clone, dump, restore and move operations –New dump tags

Current status Prototype implentation –Published as a github clone: New per volume special file for file ACLs –Reference counted entries, although 1 entry per file currently –Linked entries to track available slots Reuse “magic” field in on-disk vnode structure as an ACL pointer –known conflict with RXOSD In memory, file ACL follows the on-disk vnode structure (similar to directories) –ACL is stored and read along with the Vnode (VnLoad, VnStore) –ACL modification triggers vnode writeback

Current Status New fetchACL and storeACL RPCs defined and used –Identical signature to current RPCs –New clearACL RPC needed Client capability identifies file ACL support –Used to determine whether fileserver should restrict rights Rights restrictions not implemented yet Some security concerns whether it's acceptable to rely on capabilities to trigger this Server capability –Clients know not to assume that directory rights apply to all files –Clients use new ACL RPCs

Current Status Inherit until set semantics –Once a file ACL is set, it is “detached” from the parent - ACL changes to the parent will no longer affect it –New files have no file ACL – parent ACL applies –fetchACL returns a special value to indicate no ACL –New clearACL RPC to re-attach to parent ACL listacl – Show file ACL, or an indication that there is none – Option to show effective ACL Volume dumps –New tag identifies a file ACL –ACL retrieved from special file and added to dump if needed –On restore, insert ACL into target volume special file

What's Next Getting consensus and document protocol changes and semantics (afs3-std) RPC refresh – new ACL RPCs Consensus on on-disk structures and implementation, particularly the possible Vnode expansion Unimplemented features –Restrict rights for legacy clients –Windows client changes –Volume manipulation safeguards –Documentation changes Goal: keep the scope under control – Better chances of getting it done and integrated in a reasonable timeframe

Parting thoughts Code is easy - getting consensus is harder –Small number of key people –Few opinions, some disagreement File server code is more intuitive than cache manager code –And userspace code is easier to debug than a kernel module –But bugs can be more painful... Dependencies and links with other pending work don't help –RPC refresh, out of tree projects (RXOSD) –Other ongoing or potential projects: alternate data streams, extended attributes, etc.

Thank you Questions or comments ?