1. Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung
The Google File System
Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung

2. Paper highlights Presents a DFS tailored to a very specific workload
Mostly huge files
Mostly append-only updates
Mostly sequential reads
Client will try to order non-sequential read
Non-standard client API

3. The environment Component failures are frequent
Large system built from commodity parts
Files are huge
Multi-GB are frequent
Most files are mutated by appending
Large repositories, data streams, archival data
GFS API co-designed along with applications

4. Design assumptions (I)
Built from many cheap commodity components
Frequent failures
Modest number of very large files
100 MB or more
Two kinds or reads
Large sequential reads (100's of KB and more)
Small random reads
Often batched and sorted by applications

5. Design assumptions Many large sequential writes Append data to files
Random small writes are rare
Must implement well-defined semantics for multiple clients that concurrently append data to the same file
Sustained bandwidth is more important than latency

6. Interface Familiar POSIX API
create, delete, open, close, read and write
Two additional operations
snapshot
Creates a copy of a file or a directory
record append:
Allows multiple clients to append data to the same file at the same time
Guarantees the atomicity of updates

7. Architecture (I) Single master + multiple chunkservers
Files divided into fixed-size chunks
Each chunk has a unique immutable 64-bit chunk handle
Assigned by the master at chunk creation
Chunks are
Stored as Linux files
Replicated (default is three replicas)

8. Architecture (II) The master Maintains all metadata:
Namespace, access control, mapping from files to chunks, chunk locations
Controls all system-wide activities:
Garbage collection, chunk migration
Communicates to chunkservers through HearthBeats
Having a single master simplify the design

9. Architecture (III) Clients interact with
Master for all system metadata operations
Directly to chunkservers for all data transfers
No data caching anywhere in the system
Most applications stream through huge files
Would be ineffective
Clients cache metadata

10. Overview

11. A typical interaction
Client translates byte offset into chunk index within file
Easy because chunks have fixed-sizes
Sends request to master with file name + chunk index
Master replies with chunk handle and chunk replica locations
Typical requests are for multiple chunks

12. Chunk sizes 64 MB (that's megabytes!) Advantages
Reduces need for clients to interact with master
Many applications exhibit spatial locality
Disadvantage:
May cause hot spots if many clients have to access the same one-chunk file
Only happened for some executables
Solution was to have more replicas

13. Metadata Master stores in memory File and chunk namespaces
Mapping from files to chunks
Locations of each chunk replicas
First two are kept persistent by logging mutations on a operation log
Stored on the master's local disk
Replicated on remote hosts
Chunk replica locations are collected from chunk servers each time the master reboots

14. In-memory data structures
Make master operations fast
Let master perform periodic maintenance operations in an efficient manner
Chunk garbage collection
Re-replication after a chunkserver failure
Chunk migration to balance load and disk space
Less than 64 bytes per 64MB chunk

15. Chunk locations (I) Non-persistent Can be stored in main memory
Master polls chunkservers
At startup
Periodically after that (HeartBeat messages)
Greatly simplifies the system design
No need to keep master and chunkservers synchronized

16. Chunk locations (II) Main principle
Each chunkserver has the final word over what chunks it stores on does not sore on its local disk
Eliminates consistency issues when a chunkserver crashes or another integrates the system

17. Operation log (I) Historical record of critical metadata changes
Only persistent record of metadata
Provides a logical timeline
File chunks and their versions are identified by the logical time at which they were created
Replicated on several machines
Changes cannot be visible to clients until metadata changes ae made persistent

18. Operation log (II)
When master needs to recover the state of its file system, it will replay the log
To minimize recovery times, log must kept short
Master checkpoints its state
Each time the log reaches a maximum size
Checkpoint is a B-tree
Can be directly loaded in main memory

19. Consistency model Relaxed consistency
A file region is consistent if all clients will always see the same data on all replicas
Product of concurrent successful writes
A file region is said to be defined if
It is consistent
Clients will see the mutation writes in their entirety
Product of serial writes

20. Namespace mutations Atomic Handled exclusively by the master
Uses locks
Serial order defined by the master's operation log

21. Data mutations Can be Writes Record appends
Writes write at an application specified write offset
Record appends cause data to be appended at most once at an offset of GFS choosing
GFS may insert padding or duplicate appends

22. From the rest of the paper
Shadow masters
Additional masters that "follow" the actual master by implementing the actions recorded on the master's operation log
Not perfect mirrors
Offer read-only access to GFS

23. You are not responsible for the rest of the paper