Modular approaches In file system development The problem of creeping featurism and upgrading the old abstraction basis

Edward Shishkin edward.shishkin@gmail.com edward@redhat.com

VFSw (Sun Microsystems, 1985) Logical FS sys_mount() sys_open() sys_read() sys_write() .... VFS file_operations inode_operations address_space_operations FS VFSw: Virtual File System switches

Switches (example) Name resolution root while() { ... inode_ops = get_inode_ops(); inode_ops->lookup(); } root ext4 xfs

Extended Attributes (Xattrs) VFS FS xattrs

Journalling VFS FS xattrs journalling

Extents and delayed allocation VFS FS journalling xattrs extents delayed allocation

Tail packing VFS xattrs journalling FS extents delayed allocation

Tail<->extent conversion VFS FS journalling xattrs extents delayed allocation t-e conversion tail packing

Delayed balancing VFS xattrs journalling FS extents delayed allocation t-e conversion tail packing delayed balancing

Transparent compression VFS FS journalling xattrs extents delayed allocation t-e conversion tail packing compression delayed balancing

Transparent encryption VFS FS journalling xattrs extents delayed allocation t-e conversion tail packing compression encryption delayed balancing

Delay of actions Action What is affected external fragmentation, performance 1. Allocation internal fragmentation, performance 2. Balancing 3. Compression performance 4. Encryption performance

Sub-volumes VFS xattrs journalling FS extents delayed allocation t-e conversion tail packing compression delayed balancing encryption sub-volumes

Sub-volumes A B C D A' B' C' D'

Snapshots and WA-logging VFS FS xattrs journalling extents delayed allocation t-e conversion tail packing compression delayed balancing WA-logs encryption sub-volumes snapshots

Logging techniques Journal I. Journalling (overwrite) Permanent location New permanent location II. WA-logging (relocate)

Checksums VFS xattrs journalling FS checksums extents delayed allocation t-e conversion tail packing compression delayed balancing WA logs encryption sub-volumes snapshots

Data de-duplication VFS xattrs journalling FS checksums extents delayed allocation t-e conversion tail packing compression delayed balancing WA logs encryption sub-volumes snapshots data de-dup

Raid and LVM in FS layer FS VFS xattrs journalling checksums delayed allocation extents t-e conversion tail packing compression delayed balancing encryption WA logs sub-volumes snapshots data de-dup raid LVM

1) How to maintain? 2) How to change? VFS FS xattrs journalling checksums delayed allocation extents t-e conversion tail packing compression delayed balancing encryption COW sub-volumes snapshots data de-dup raid LVM

Possible ways to implement a new feature in the FS layer: Write a new FS from scratch Add this feature to an existing FS Possible ways to implement a new feature in the FS layer:

Cost of a modern local FS Kernel module: 25 human years Fsck and user-space utilities: 8 human years Conclusion: We should refuse the first way: writing FS from scratch is too expensive (and the cost tends to increase with years)!

How FS should be designed to minimize overheads when adding a new feature? To answer on this question we need to possess a deep understanding of what a file system feature is. And some progress in this direction is possible with 2 different approaches: Approach #1: DAGI, plugins (Reiser4) Approach #2: Translators (GlusterFS)

Introducing a new interface is not overdesign if at least one of the following takes place: encapsulating disk formats switches transitions chances that it will be used by other subsystems

VFS TREE ->search_by_key(); ->insert_object(balance_flags); ->cut_object(balance_flags); ->balance(); FS TREE

VFS NODE ->lookup(); ->create_item(); ->delete_item(); ->shift(); ... FS TREE ITEM ->lookup(); ->number_of_units(); ->mergeable(this, neighbor); ->read(); ->write(); ->read_pages(); ... ITEM NODE

Managing transactions VFS TXM ->init_context(); ->exit_context(); ->capture_block(); ... FS TXM TREE ATOM ATOM ->mark_block_dirty(); ->flush(); ->commit(); ... ITEM NODE Managing transactions

Delayed actions VFS SPACE ->alloc_extent(); ->free_extent(); ... TXM TREE ATOM COMPRESS ->compress(); ->decompress(); ... ITEM NODE COMPRESS SPACE Delayed actions

Snapshots SNPM ->create_subvolume(); ... VFS FS TXM SNPM TREE ATOM ITEM NODE NODE COMPRESS SPACE Snapshots

Trivially graded DAGI-Graded (old abstraction, Sun 1985) (Namesys, 2004) VFS VFS FS FS TXM SNPM TREE ATOM ITEM NODE COMPRESS SPACE DAGI: Direct Acyclic Graph of Interfaces with weak CS relationships

Polymorphism of zero order X Polymorphism of zero order C Z x->y Y A C1 X C2 Polymorphism of first order Z C3 x->x z->x z->y Y B->A

GlusterFS: Client Server chains of interfaces with X1 VFS X2 Client FS ... Xn Xn+1 Server GlusterFS: chains of interfaces with strong CS-relationships ...