1. Distributed File System By Manshu Zhang

2. Outline Basic Concepts Current project Hadoop Distributed File System Future work Reference

3. DFS A distributed implementation of the classical time sharing model of a file system, where multiple users share files and storage resources.

4. Key Characteristics of DFS Dispersion Clients and files Multiplicity Clients and files

5. Primary issues of DFS Naming and Transparency Fault Tolerance

6. Naming Naming – mapping between logical and physical objects. Multilevel mapping. Transparent replicas and location

7. Naming Schemes — Three Main Approaches Host name + local name  guarantees a unique system wide name. Mount remote directories to local directories  once mounted, files can be referenced in a location-transparent manner Total integration of the component file systems.  A single global name structure  If a server is unavailable, some arbitrary set of directories on on different machines also becomes unavailable

8. Transparency(1) Login Transparency: User can log in at any host with uniform login procedure and perceive a uniform view of the file system. Access Transparency: Client process on a hots has uniform mechanism to access all files in system regardeless of files are on local/remote host. Location Transparency: The names of the files do not reveal their physical location.

9. Transparency(2) Concurrency Transparency: An update to a file should not have effect on the correct execution of other process that is concurrently sharing a file. Replication Transparency: Files may be replicated to provide redundancy for availability and also to permit concurrent access for efficiency.

10. Fault Tolerance Stateful Vs. Stateless  Maintain information on client File Replication

11. Distinctions Between Stateful & Stateless Service Failure Recovery.  A stateful server loses all its volatile state in a crash.  With stateless server, the effects of server failure and recovery are almost unnoticeable.

12. File Replication Several copies of a file's contents at different locations enable multiple servers to share the load of providing the service Naming scheme maps a replicated file name to a particular replica. Updates

13. Current Project HDFS: Hadoop Distributed File System Distributed parallel fault tolerant file system. It is designed to reliably store very large files across machines in a large cluster. Efficient, reliable, and open source

15. Naming: central metadata server Synchronization: write-once-read-many, give locks on objects to clients, using leases Consistency and replication: server side replication, asynchronous replication, checksum Fault tolerance: failure as norm Security: no dedicated security mechanism

16. Future Work Robustness of data sharing model The preceding section, architecture, naming, synchronization, availability, heterogeneity and support for databases Security

17. Reference [1] Thanh, T.D.; Mohan, S.; Choi, E.; SangBum Kim; Pilsung Kim. 2008Networked Computing and Advanced Information Management. “A Taxonomy and Survey on Distributed File Systems” [2] Randy chow,1997,Distributed operating systems & Algorithms [3] Eliezer Levy, Abraham Silberschatz. December 1990 Computing Surveys (CSUR), Volume 22 Issue 4. ”Distributed file systems: concepts and examples”. [4]http://hadoop.apache.org/common/docs/current/hdfs_design.html#Introd uction [4]http://hadoop.apache.org/common/docs/current/hdfs_design.html#Introd uction [5]http://www.snia.org/events/wintersymp2009/cloud/dhruba_hadoop_snia. pdf [5]http://www.snia.org/events/wintersymp2009/cloud/dhruba_hadoop_snia. pdf

18. [6]http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_file_s ystems [6]http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_file_s ystems [7]http://en.wikipedia.org/wiki/Hadoop#Hadoop_Distributed_File_Sy stem [7]http://en.wikipedia.org/wiki/Hadoop#Hadoop_Distributed_File_Sy stem [8]http://www.cs.gsu.edu/~cscyqz/courses/aos/slides08/ch6.1- Fall08.pptx [8]http://www.cs.gsu.edu/~cscyqz/courses/aos/slides08/ch6.1- Fall08.pptx

19. Q&A?

20. Thank you!