The Design and Implementation of a Log-Structured File System Presented by Carl Yao

Main Ideas Memory becomes cheaper, file systems use bigger buffer caches in memory, most reads don't go to disk, most disk accesses are writes Regular data writes can be delayed a little bit, at the risk of losing some updates Meta data writes cannot be delayed, because risk is too high Results: most disk accesses are meta data writes FFS uses "update-in-place," spends lot of time seeking meta data and regular data on disk, causing low disk bandwidth usage LFS gives up "update-in-place," writes new copy of updates together – Advantage: Writing is fast (main problem of FFS solved) – Disadvantage: Complexity in reading (but cache relieves this problem), overhead in segment cleaning

Technology Trend Processor speed improving exponentially Memory capacity improving exponentially Disk capacity improving exponentially – But, not transfer bandwidth and seek times Transfer bandwidth can be improved with RAID Seek times hard to improve

Problems with Fast File System Problem 1: File information is spread around the disk – inodes are separate from file data – 5 disk I/O operations required to create a new file directory inode, directory data, file inode (twice for the sake of disaster recovery), file data Results: less than 5% of the disk’s potential bandwidth is used for writes Problem 2: Meta data updates are synchronous application does not get control until completion of I/O operation

Solution: Log-Structured File System Improve write performance by buffering a sequence of file system changes to disk sequentially in a single disk write operation. Logs written include all file system information, including file data, file inode, directory data, directory inode.

Simply Example of LFS

File Location and Reading Still uses FFS’s inode structure. But inodes are not located at fixed positions. Inode map is used to locate a file’s latest version of inode. Inode map itself is located in different places of the disk, but its latest version is loaded into memory for fast access. This way, file reading performance of LFS is similar to FFS. (Really?)

File Reading Example Pink: file data Green: inode Brown: inode map (written to logs but loaded in memory)

File Writing Performance Improved

Reclaiming Space in Log Eventually, the log reaches the end of the disk partition – so LFS must reuse disk space deleted files overwritten blocks – space can be reclaimed in the background or on- demand – goal is to maintain large free extents on disk

Two Approaches to Reclaim Space Problem with threaded log—fragmentation Problem with copy and compact—cost of copying data

Sprite LFS’ Solution: Combination of Both Approaches Combination of copying and threading – divide disk up into fixed-size segments – copy live blocks to free segments - try to collect long-lived data (not accessed for a while) permanently into segments – Log is threaded on a segment-by-segment basis

Segment Cleaning Cleaning a segment – read several segments into memory – identify the live blocks – write live data back (hopefully into a smaller number of segments) How are live blocks identified? – each segment maintains a segment summary block to identify what is in each block and which inode this block belongs to – crosscheck blocks with owning inode’s block pointers

Segment Cleaning Policy When to clean? – Sprite starts cleaning when number of clean segments drops below a threshold (say 50 segments). How many segments to clean? – A few tens of segments at a time until the number of clean segments surpasses another threshold (say 100 segments) Which segments to clean? – cleaning segments with little dead data gives little benefit – want to arrange it so that most segments have good utilization, and the cleaner works with the few that don’t – how should one do this?

Which Segments to Clean? Two kinds of segments – hot segments: very frequently accessed however, cleaning them yields small gains – cold segments: very rarely accessed cleaning these yields big gains because it will take a while for it to reaccumulate unused space U = utilization; A = age (most recent modified time of any block in the segment); Benefit to cost = (1–U)*A/(U+1) – Pick the segment that maximizes the above ratio – This policy reaches a sweet spot where reusable blocks in cold segments are frequently cleaned, while those in hot segments are infrequently cleaned

Segment Cleaning Result The disk becomes a bimodal segment distribution: – Most of the segments are nearly full – A few are empty or nearly empty – The cleaner can almost always work with the empty segments

Crash Recovery Crash in UNIX is a mess – disk may be in inconsistent state e.g., middle of file creation, file created but directory not updated – running fsck takes a long time Not a mess in LFS – just look at end of log; scan backward to last consistent state

Checkpoints A checkpoint is a position in the log where all file systems structures are consistent Creation of a checkpoint: – 1. Write out all modified info to log, including metadata – 2. Write checkpoint region to special place on disk On reboot, read checkpoint region to initialize main- memory data structures – use 2 checkpoints in case checkpoint write crashes!

Roll-Forward Try to recover as much data as possible Look at segment summary blocks – if new inode and data blocks, but no inode map entry, then update inode map; new file is now integrated into file system – if only data blocks, then ignore Need special record for directory change – this avoid problems with inode written, but directory not written – appears before the corresponding directory block or inode – again, roll-forward

Test Results Sprite LFS clearly beat SunOS in small-file read and write performance Sprite LFS beat SunOS in large-file writing, made a draw with SunOS in large-file reading, lost to SunOS in reading a file sequentially after it has been written randomly. – In the last case, LFS lost because it requires seeks, but SunOS does not.