1. Data Management with Google File System Pramod Bhatotia wp. mpi-sws
Data Management with Google File System Pramod Bhatotia wp.mpi-sws.org/~bhatotia
Distributed Systems, 2016

2. From the last class: How big is “Big Data”?
processes 20 PB a day (2008)
crawls 20B web pages a day (2012)
>10 PB data, 75B DB calls per day (6/2012)
>100 PB of user data TB/day (8/2012)
S3: 449B objects, peak 290k request/second (7/2011)
1T objects (6/2012)

3. Distributed systems for “Big Data” management
Bandwidth: ~180 MB/sec,
Read time 1PB: ~65 days!
Single disk
Multiple disks
Aggregate bandwidth:
~ No. of disks*180 MB/sec

4. In today’s class How to manage “Big data” in distributed setting?
Google File System (GFS)
Open source: Hadoop Distributed File System (HDFS)

5. Distributed file system
From Google search:
“A distributed file system is a client/server-based application that allows clients to access and process data stored on the server as if it were on their own computer.”
E.g.: NFS, AFS
Network
Read File
Data
Client
Server

6. Key design requirements
Large files
Scalable
Performance
Write once (or append only)
Reliable
Available
Namespace
Concurrency

7. Beauty of GFS/HDFS
MapReduce
Applications
GFS/HDFS

8. Interface/API Simple shell interface
$ hadoop dfs -<fs commands> <arguments>
<fs commands>: mv, cp, chmod, copyFromLocal, copyToLocal, rm, du, etc.
Additional commands for snapshots, appends
Similar programming APIs to access HDFS
Not compatible w/ POSIX API; HDFS only allows appends

9. Distributed file system: GFS/HDFS
Architecture
Master node
(NameNode)
Namespace
ACL
Data management
Lease management
Garbage collection
Reliability of chunks
Distributed file system: GFS/HDFS

10. Architecture Master node (NameNode) Data Node Data Node Data Node Data

11. Basic functioning Metadata Namenode (metadata) Client
(caches metadata)
Input file
Data (read/write)
Data
Node
Data
Node

12. Read operation Metadata (red chunk) Master node (NameNode) Client
Read red chunk
Data
Node
Data
Node

13. Write operation Metadata (red chunk) Master node (NameNode) Client B C
Send data to buffer A B
Data
Node
Data
Node
Data
Node

14. Using lease management by the master
Commit order
Using lease management by the master
Primary
(Replica A)
Secondary
(Replicas B & C) C A B
Data
Node
Data
Node
Data
Node
Update buffer at A
U1, U2, U3
Update buffer at B
U3, U2, U1
Update buffer at C
U1, U3, U2

15. Primary commit order Client Primary (Replica A) Secondary
(Replicas B & C)
Write
complete C A B
Data
Node
Data
Node
Data
Node
U1, U2, U3
U1, U2, U3
U1, U2, U3
Commit
order
Commit
order
Commit
order

16. Consistency semantics
Updates (only for GFS)
File region may end up containing mingled fragments from different clients
E.g., writes to different chunks may be ordered differently by their different primary replica
Thus, writes are consistent but undefined in GFS
Appends
Append causes data to be appended atomically at least once
Offset chosen by HDFS, not by the client

17. Discussion: Other design details
Chunk replication: placement and re-balancing
Namespace management & locking
Garbage collection (lazy)
Data integrity
Snapshots (using copy-on-write)
Master replication

18. References GFS [SOSP’03] Tachyon [SoCC’13] TidyFS [USENIX ATC’11]
Original Google File System paper
Tachyon [SoCC’13]
Distributed in-memory file system for Spark
TidyFS [USENIX ATC’11]
A distributed file system by Microsoft
Resources:

19. Pramod Bhatotia wp.mpi-sws.org/~bhatotia
Thanks!
Pramod Bhatotia wp.mpi-sws.org/~bhatotia