A Low-Bandwidth Network File System A. Muthitacharoen, MIT B. Chen, MIT D. Mazieres, NYU

Key Ideas A network file systems for slow or wide-area networks Exploits similarities between files or versions of the same file Avoids sending data that can be found in the server’s file system or the client’s cache Also uses conventional compression and caching Requires 90% less bandwidth than traditional network file systems

Working on slow networks Make local copies Must worry about update conflicts Use remote login Only for text-based applications Use instead a LBFS Better than remote login Must deal with issues like auto-saves blocking the editor for the duration of transfer

LBFS Exploits cross-file similarities especially with previous versions of the same file Auto-save files, … LBFS file server divides the files it stores into chunks and indexes the chunks by hash value LBFS client similarly indexes a large persistent file cache LBFS never transfers chunks that the recipient already has

Previous Work (I) AFS Callbacks require server to notify clients when a cached file has been modified Leases achieve same goal but have an expiration time Coda supports slow networks and even disconnected operation Defers some updates to saves bandwidth OceanStore applies Bayou’s conflict resolution mechanisms to a file system

Previous Work (II) Operation-based updates (Lee et al.) Proxy-client close to the server duplicates client computations in the hope of duplicating its output files Spring and Wetherall propose to use two large cooperating caches storing identical copies of the last n megabytes of network traffic Rsync uses directory tree mirroring at client and server.

LBFS LBFS provides close-to-open consistency Similar to AFS session consistency LBFS assumes clients will have a cache large enough to contain a user’s entire working set of files When possible, LBFS reconstitutes files using chunks of existing data in the file system and client cache instead of transmitting those chunks over the network

Indexing Issues Major challenge is keeping the index a reasonable size while dealing with shifting offsets Indexing conventional file blocks would not work Indexing and hashing overlapping file blocks at all offsets would require too much space

LBFS Solution Considers only non-overlapping chunks of files Sets chunk boundaries based on file contents to avoid sensitivity to shifting file offset Examines every overlapping 48-byte region of the file to selects boundary regions, or breakpoints, using Rabin fingerprints Expected chunk size is 8 KB plus the size of the 48-byte breakpoint window

Handling Insertions

More Indexing Issues Pathological cases Very small chunks Sending hashes of chunks would consume as much bandwidth as just sending the file Very large chunks Cannot be sent in a single RPC LBFS imposes minimum and maximum chuck sizes

The Chunk Database Indexes each chunk by the first 64 bits of its SHA-1 hash To avoid synchronization problems, LBFS always recomputes the SHA-1 hash of any data chunk before using it Simplifies crash recovery Recomputed SHA-1 values are also used to detect hash collisions in the database

Protocol Based on NFS version 3 Adds Extensions to exploit inter-file commonality (GETHASH) Leases Compresses all traffic using conventional gzip

File Consistency (I) Whenever a client makes any RPC on an LBFS file, it gets back a read lease on the file. If a user opens a file whose lease has expired, the client asks the server for the attributes of the file Grants the client a lease on the file. Client can check if it has the current version of the file in its cache If the file times have changed, client must obtain new contents of file from server

File Consistency (II) No need for write leases LBFS provides close-to-open consistency Server never demands back a dirty file If multiple clients are writing the same file,the last one to close the file will overwrite changes from the others File updates are atomic Limits damage caused by concurrent updates

Security Issues LBFS uses SFS security infrastructure Servers have public keys Messages are encrypted Specific security issue: A user could check whether the file system contains a particular chunk of data by observing subtle timing differences in server’s answer to CONDWRITE request

Implementation (I)

Implementation (II) Uses NFS Two NFS-related issues When server commits a temporary file to a target file, it must copy the contents of the temporary file onto the target file to preserve the target file i-node Hard to preserve previous contents of a truncated file Message order is guaranteed by TCP

Evaluation (I) Communality of data in /usr/local

Evaluation (II) Normalized bandwidth consumption (2 of 3 benchmarks)

Key First four bars of each workload show upstream bandwidth, the second four downstream bandwidth. CIFS is Windows natural network file system “Leases+Gzip” uses LBFS file caching, leases, and data compression but not its chunking scheme “LBFS, new DB” is LBFS starting with a a new database

Evaluation (III) Normalized application times

Key Execution times weere normalized orma,ized execution times Measurements made over a cable modem link with 384 Kb/sc uplink and 1.5 Mb/s downlink LAN data were obtained on a 100 Mb/s full- duplex LAN.

Conclusion Under normal circumstances, LBFS consumes 90% less bandwidth than traditional file systems. Makes transparent remote file access a viable and less frustrating alternative to running interactive programs on remote machines.