1. Ivy: A Read/Write Peer-to-Peer File System A. Muthitacharoen, R. Morris, T. M. Gil, and B. Chen In Proceedings of OSDI ‘02 2003-4-29 Presenter : Chul Lee

2. What is IVY? A multi-user read/write peer-to-peer file system No centralized/dedicated components Single file system image Conventional file system interface –Case study of DHT use!

3. Ivy uses DHT DHT provides –Simple API Put(key, value) and get(key)  value –Availability (Replication) –Robustness (Integrity checking) Distributed hash table Distributed application get (key) data put(key, data) Lookup service lookup(key)node IP address (Ivy) (DHash) (Chord)

4. Prob.: Shared Data w/ DHT Root Inode Directory Block File3 Inode File2 Inode File1 Inode File3 Data Internet DHT node

5. Challenges Consistency of file system meta-data Locking is an unattractive approach over unreliable participants. Undo modifications by untrustworthy participants Operate while partitioned, repair conflicting updates

6. Solution: Log Based Update: Each participant maintains a log of changes to the file system Lookup: Each participant scans all logs

7. Software Structure Local NFS loop-back server App NFS Client Ivy Server DHT Node DHT Node DHT Node kernel user system calls NFS RPCs Internet DHT Node

8. Example: Using Log Local NFS ClientLocal Ivy Server LOOKUP(“d”, I-Num=10) I-Num=1000 CREATE(“aaa”, I-Num=1000) I-Num=9956 WRITE(“hello”, 0, I-Num=9956) OK echo hello > d/aaa LOOKUP finds the I-Number of directory “d” CREATE creates file “aaa” in directory “d” WRITE writes “hello” at offset 0 in file “aaa”

9. Using Log: File Creation Type: Create I-num: 9956 Type: Link Dir I-num: 1000 File I-num: 9956 Name: “aaa” Type: Write I-num: 9956 Offset: 0 Data: “hello” … Log Head A log record describes a change to the file system

10. Using Log: Lookup Type: Link Dir I-num: 1000 File I-num: 9956 Name: “aaa” Type: Link Dir I-num: 1000 File I-num: 9876 Name: “bbb” Type: Remove Dir I-num: 1000 Name: “aaa” A scan follows the log backwards in time LOOKUP(name, dir I-num): last Link, but stop at Remove READDIR(dir I-num): accumulate Links, minus Removes

11. Contributions Multi-user read/write peer-to-peer storage system Distributed file system with useful integrity properties based on untrusted components Use of distributed hash tables as a building block

12. Design DHash – maps keys to arbitrary values Log Data Structure – a linked list View – a set of logs Combining logs – in ordering records Snapshot – state of the file system

13. Log Data Structure A linked list of immutable log records

14. Log record types Roughly NFS update operations 160-bit i-numbers as file handle

15. User Cooperation: Views Set of logs that comprise the file system View block –a immutable DHash content-hash block

16. Combining Logs Ivy orders records using version vectors Seq. field – starts from zero for each log Version vector: tuple (U:V) for each log –U: Dhash key of the log-head –V: Sequence number of the most recent record Example: (A:5 B:7) –< (A:6 B:7) BUT concurrent with (A:6 B:6) Public keys used to order in case of concurrency

17. Snapshots Each Ivy participant constructs a private snapshot for speed Contains the entire state of the file system Each snapshot stored in DHash for persistency as content-hash blocks

18. Snapshot Data Structure

19. Application Semantics Concurrent Updates Partitioned Updates/ Conflict Resolution

20. Concurrent Updates Ivy does not serialize all updates Problem –Unlink( “ a ” ) and rename( “ a ”, “ b ” ) at same time –Ivy correctly lets only one take effect –But it may return “ success ” status for both

21. Partitioned Updates Ivy is not directly aware of partitions –Ivy’s design maximizes availability at the expense of consistency –Letting updates proceed in all partitions All updates during a partition are concurrent updates Conflict resolution -> “lc” tools

22. WAN Evaluation on MAB Modified Andrew Benchmark 4 DHash nodes Round-trip times: 9, 16, 82 milliseconds No DHash replication 4 logs One active writer

23. WAN Performance PhaseIvyNFS Mkdir11.24.8 Write89.242.0 Stat65.647.8 Read65.855.6 Compile144.2130.2 Total376.0280.4

24. Summary Exploring use of DHTs as a building block Case study of DHT use: Ivy –Read/write peer-to-peer file system Suitable for small groups of cooperating participants who do not have a single central server

25. Critiques Indefinite logs Scanning all logs for each request Rely on DHT ’ s block availability and robustness

26. Discussion DHT interface ~ Disk Sector R/W interface Performance vs. Semantic Any other applications of DHT –DB, LDAP server…