1 Chap8 Distributed File Systems  Background knowledge  8.1Introduction –8.1.1 Characteristics of File systems –8.1.2 Distributed file system requirements.

Slides:



Advertisements
Similar presentations
DISTRIBUTED FILE SYSTEMS
Advertisements

From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, © Addison-Wesley 2012 Slides for Chapter 12: Distributed.
Yanjun Zhao.  A network file system where a single file system can be distributed across several physical computers  allows administrators to group.
Using DSVM to Implement a Distributed File System Ramon Lawrence Dept. of Computer Science
CS 425 / ECE 428 Distributed Systems Fall 2014 Indranil Gupta (Indy) Lecture 27: Distributed File Systems All slides © IG.
Slides for Chapter 8: Distributed File Systems
Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition File-System Interface.
CS Distributed Computing Systems Chapter 8: Distributed File Systems Chin-Chih Chang, From Coulouris, Dollimore and Kindberg Distributed.
1 Reliable Distributed Systems Stateless and Stateful Client- Server Systems.
Chapter 10: File-System Interface
6/24/2015B.RamamurthyPage 1 File System B. Ramamurthy.
1 Course Outline Processes & Threads CPU Scheduling Synchronization & Deadlock Memory Management File Systems & I/O Networks, Protection and Security.
NFS. The Sun Network File System (NFS) An implementation and a specification of a software system for accessing remote files across LANs. The implementation.
7/15/2015B.RamamurthyPage 1 File System B. Ramamurthy.
Distributed Computing Systems
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Google∗
CSE 486/586, Spring 2012 CSE 486/586 Distributed Systems Distributed File Systems Steve Ko Computer Sciences and Engineering University at Buffalo.
1 The Google File System Reporter: You-Wei Zhang.
CSC 456 Operating Systems Seminar Presentation (11/13/2012) Leon Weingard, Liang Xin The Google File System.
Networked File System CS Introduction to Operating Systems.
Copyright © George Coulouris, Jean Dollimore, Tim Kindberg This material is made available for private study and for direct.
Distributed File Systems
Distributed File Systems
Copyright © George Coulouris, Jean Dollimore, Tim Kindberg This material is made available for private study and for direct.
Lecture 7: Distributed File Systems Haibin Zhu, PhD. Assistant Professor Department of Computer Science Nipissing University © 2002.
Distributed system Distributed File System Nguyen Huu Tuong Vinh Huynh Thi Thu Thuy Dang Trang Tri.
Distributed File Systems Overview  A file system is an abstract data type – an abstraction of a storage device.  A distributed file system is available.
Chap 10 File-System Interface. Objectives To explain the function of file systems To describe the interfaces to file systems To discuss file-system design.
What is a Distributed File System?? Allows transparent access to remote files over a network. Examples: Network File System (NFS) by Sun Microsystems.
Chapter 10: File-System Interface Silberschatz, Galvin and Gagne ©2005 Operating System Concepts – 7 th Edition, Jan 1, 2005 Chapter 10: File-System.
Page 110/19/2015 CSE 30341: Operating Systems Principles Chapter 10: File-System Interface  Objectives:  To explain the function of file systems  To.
Advanced Computer Networks Topic 2: Characterization of Distributed Systems.
Introduction to DFS. Distributed File Systems A file system whose clients, servers and storage devices are dispersed among the machines of a distributed.
DISTRIBUTED FILE SYSTEM 1 DISTRIBUTED FILE SYSTEMS From Chapter 8 of Distributed Systems Concepts and Design,4 th Edition, By G. Coulouris, J. Dollimore.
CSE 486/586 CSE 486/586 Distributed Systems Distributed File Systems Steve Ko Computer Sciences and Engineering University at Buffalo.
DISTRIBUTED FILE SYSTEMS Pages - all 1. Topics  Introduction  File Service Architecture  DFS: Case Studies  Case Study: Sun NFS  Case Study: The.
Computer Science Lecture 19, page 1 CS677: Distributed OS Last Class: Fault tolerance Reliable communication –One-one communication –One-many communication.
Lecture 27-1 Computer Science 425 Distributed Systems CS 425 / ECE 428 Fall 2013 Indranil Gupta (Indy) December 3, 2013 Lecture 27 Distributed File Systems.
Distributed System and Middleware Distributed Systems Distributed File System Dr. Sunny Jeong. Mr. Coling Zhang
DISTRIBUTED FILE SYSTEM 1 DISTRIBUTED FILE SYSTEMS.
Lecture 27-1 Lecture 28-1 Computer Science 425 Distributed Systems CS 425 / CSE 424 / ECE 428 Fall 2012 Indranil Gupta (Indy) December 6, 2012 Lecture.
1 Reliable Distributed Systems Stateless and Stateful Client- Server Systems.
14.1 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Chapter 10 & 11: File-System Interface and Implementation.
It consists of two parts: collection of files – stores related data directory structure – organizes & provides information Some file systems may have.
1 Distribuerede systemer og sikkerhed – 4. marts 2002 zFrom Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design zEdition 3, © Addison-Wesley.
© Oxford University Press 2011 DISTRIBUTED COMPUTING Sunita Mahajan Sunita Mahajan, Principal, Institute of Computer Science, MET League of Colleges, Mumbai.
Distributed File Systems Questions answered in this lecture: Why are distributed file systems useful? What is difficult about distributed file systems?
Distributed File System. Outline Basic Concepts Current project Hadoop Distributed File System Future work Reference.
W4118 Operating Systems Instructor: Junfeng Yang.
COT 4600 Operating Systems Spring 2011 Dan C. Marinescu Office: HEC 304 Office hours: Tu-Th 5:00 – 6:00 PM.
Computer Science Lecture 19, page 1 CS677: Distributed OS Last Class: Fault tolerance Reliable communication –One-one communication –One-many communication.
DISTRIBUTED FILE SYSTEM- ENHANCEMENT AND FURTHER DEVELOPMENT BY:- PALLAWI(10BIT0033)
1 Reliable Distributed Systems Stateless and Stateful Client- Server Systems Based on K. Birman’s of Cornell, Dusseu’s of Wisconsin.
Distributed Systems Course Distributed File Systems
Distributed File Systems (DFS)
Lecture 25: Distributed File Systems
Lecture 25: Distributed File Systems
File System B. Ramamurthy B.Ramamurthy 11/27/2018.
Slides for Chapter 8: Distributed File Systems
Distributed File Systems
Distributed File Systems
Distributed Systems Course Distributed File Systems
Distributed File System
Lecture 25: Distributed File Systems
Distributed file system
Distributed File Systems
Outline Review of Quiz #1 Distributed File Systems 4/20/2019 COP5611.
Distributed File Systems
Lecture 4: File-System Interface
Introduction to Operating Systems
Presentation transcript:

1 Chap8 Distributed File Systems  Background knowledge  8.1Introduction –8.1.1 Characteristics of File systems –8.1.2 Distributed file system requirements  8.2 File service architecture

2 Background Knowlege  File is a collection of data with a user view (file structure) and a physical view (blocks)  File systems implement file management:  Naming and locating a file  Accessing a file – create, delete, open, close, read, write, append, truncate  Physical allocation of a file.  Security and protection of a file.  A distributed file system (DFS) is a file system with distributed storage and users.  DFS provides transparency of location, access, and migration of files.  DFS systems use cache replicas for efficiency and fault tolerance

3 8.1 Introduction  A distributed file system enables programs to store and access remote files exactly as they do local ones.  Two distributed systems that have been in widespread used for a decade or more. –Sun Network File System (NFS) –the Andrew File System (AFS)

4 Figure 8.1 Storage systems and their properties SharingPersis- tence Distributed cache/replicas Consistency maintenance Example Main memory RAM File systemUNIX file system Distributed file systemSun NFS Web Web server Distributed shared memoryIvy (Ch. 16) Remote objects (RMI/ORB)CORBA Persistent object store 1 CORBA Persistent Object Service Persistent distributed object store PerDiS, Khazana * Types of consistency between copies:1 - strict one-copy consistency √ - approximate consistency X - no automatic consistency

Characteristics of file systems File Attribute Record Block length creation timestamp read timestamp write timestamp attribute timestamp reference count version number ownership access control list Directory Module File Module Access control Module Read & Write Module Block Module Device Module File System Modules

6 Figure 8.2 File system modules File system modules *

7 updated by system: File length Creation timestamp Read timestamp Write timestamp Attribute timestamp Reference count Owner File type Access control list E.g. for UNIX: rw-rw-r-- File attribute record structure * Figure 8.3 File attribute record structure updated by owner:

8 File system operation * filedes = open(name, mode) filedes = creat(name, mode) Opens an existing file with the given name. Creates a new file with the given name. Both operations deliver a file descriptor referencing the open file. The mode is read, write or both. status = close(filedes)Closes the open file filedes. count = read(filedes, buffer, n) count = write(filedes, buffer, n) Transfers n bytes from the file referenced by filedes to buffer. Transfers n bytes to the file referenced by filedes from buffer. Both operations deliver the number of bytes actually transferred and advance the read-write pointer. pos = lseek(filedes, offset, whence) Moves the read-write pointer to offset (relative or absolute, depending on whence). status = unlink(name)Removes the file name from the directory structure. If the file has no other names, it is deleted. status = link(name1, name2)Adds a new name (name2) for a file (name1). status = stat(name, buffer)Gets the file attributes for file name into buffer. Figure 8.4 UNIX file system operations

9 Tranparencies Access:Same operations Location:Same name space after relocation of files or processes Mobility:Automatic relocation of files is possible Performance: Satisfactory performance across a specified range of system loads Scaling:Service can be expanded to meet additional loads Concurrency properties Isolation File-level or record-level locking Other forms of concurrency control to minimise contention Replication properties File service maintains multiple identical copies of files Load-sharing between servers makes service more scalable Local access has better response (lower latency) Fault tolerance Full replication is difficult to implement. Caching (of all or part of a file) gives most of the benefits (except fault tolerance) Heterogeneity properties Service can be accessed by clients running on (almost) any OS or hardware platform. Design must be compatible with the file systems of different OSes Service interfaces must be open - precise specifications of APIs are published. Fault tolerance Service must continue to operate even when clients make errors or crash. at-most-once semantics at-least-once semantics requires idempotent operations Service must resume after a server machine crashes. If the service is replicated, it can continue to operate even during a server crash. Consistency Unix offers one-copy update semantics for operations on local files - caching is completely transparent. Difficult to achieve the same for distributed file systems while maintaining good performance and scalability. Security Must maintain access control and privacy as for local files. based on identity of user making request identities of remote users must be authenticated privacy requires secure communication Service interfaces are open to all processes not excluded by a firewall. vulnerable to impersonation and other attacks Efficiency Goal for distributed file systems is usually performance comparable to local file system Distributed File service requirements  Transparency  Concurrency  Replication  Heterogeneity  Fault tolerance  Consistency  Security  Efficiency.. *

File service architecture Client computerServer computer Application program Application program Client module Flat file service Directory service Lookup AddName UnName GetNames Read Write Create Delete GetAttributes SetAttributes * Figure 8.5

File service architecture.  An abstract model with flat file, directory and client services:  Flat file service: concerned with implementing operaions on the contents of files. Unique file identifiers(UFIDS) are used to refer to files in all requests. implements create, delete, read, write, get attribute, set attribute and access control operations.  Directory service: Creates and updates directories ( hierarchical file structures) and provides mappings between user names of files and the unique file ids in the flat file structure.  Client Service: runs in each client’s computer, integrating and expanding flat file and directory services to provide transparency, caching, etc.

12 Server operations for the model file service Flat file service Read(FileId, i, n) -> Data Write(FileId, i, Data) Create() -> FileId Delete(FileId) GetAttributes(FileId) -> Attr SetAttributes(FileId, Attr) Directory service Lookup(Dir, Name) -> FileId AddName(Dir, Name, File) UnName(Dir, Name) GetNames(Dir, Pattern) -> NameSeq

13 File Group A collection of files that can be located on any server or moved between servers while maintaining the same names. –Similar to a UNIX filesystem –Helps with distributing the load of file serving between several servers. –File groups have identifiers which are unique throughout the system (and hence for an open system, they must be globally unique).  Used to refer to file groups and files To construct a globally unique ID we use some unique attribute of the machine on which it is created, e.g. IP number, even though the file group may move subsequently. IP address date 32 bits 16 bits File Group ID: *