1. Distributed Operating Systems CS551 Colorado State University at Lockheed-Martin Lecture 8 -- Spring 2001

2. 4 April 2001CS-551, Lecture 82 CS551: Lecture 8 n Topics – Distributed File Systems (Chapter 8) n Distributed Name Service n Distributed File Service n Distributed Directory Service n NFS n X.500 – Distributed Synchronization (Chapter 10) n Global Time n Physical Clocks n Network Time Protocol (NTP) n Logical Clocks

3. 4 April 2001CS-551, Lecture 83 Definitions n DFSs “support the sharing of information in the form of files throughout an intranet. A well-designed file service provides access to files stored at a server with performance and reliability similar to … files stored on local disks. A distributed file system enables programs to store and access remote files exactly as they do local ones, allowing users to access files from any computer in an intranet.” (Coulouris, Dollimore, Kindberg, 2001)

4. 4 April 2001CS-551, Lecture 84 Definitions, continued n “…in a DS, it is important to distinguish between the concepts of the file service and the file server. The file service is the specification of what the file system offers to its clients … the file system’s interface to the clients. A file server, in contrast, is a process that runs on some machine and helps implement the file service. A system may have one file server or several.” (Tanenbaum, 1995)

5. 4 April 2001CS-551, Lecture 85 Upload/Download Model ServerClient Client’s copy Updated File Original File Adapted from Tanenbaum (1995)

6. 4 April 2001CS-551, Lecture 86 Remote Access Model ServerClient Client requests access from remote file File does not move Adapted from Tanenbaum (1995)

7. 4 April 2001CS-551, Lecture 87 Terms n File system – “an abstract view of secondary storage” – “responsible for n Global naming n File access n Overall file organization” n Distributed Name Service – “focuses on the issues related to filenames”

8. 4 April 2001CS-551, Lecture 88 Basic File Systems n File Storage – Structured versus non-structured n File Attributes – File name, size, owner, creation/modification dates, version, protection information n File Protection Modes – Read, write, execute, append, truncate, delete

9. 4 April 2001CS-551, Lecture 89 Figure 8.4 Structured versus Unstructured Files.

10. 4 April 2001CS-551, Lecture 810 Figure 8.5 Access Matrix.

11. 4 April 2001CS-551, Lecture 811 Figure 8.6 Access List for File 1.

12. 4 April 2001CS-551, Lecture 812 Goals of a DFS n Network Transparency – Looks like a traditional file system on a mainframe – User need not know a file’s location n High Availability – Users should have easy access to files, wherever the users or files are located – Tolerant of failures

13. 4 April 2001CS-551, Lecture 813 Architecture n On the Network – File servers: hold the files – Clients: make accesses to the servers n Name Server (does name resolution) – Maps names to directories/files n Cache Manager – Implements file caching – Often at both server and clients – Coordinates to avoid inconsistent file copies

14. 4 April 2001CS-551, Lecture 814 Mechanisms of a DFS n Mounting – Binding together of different filename spaces to form a single name space – A name space is mounted to (or bounded to) a mount point (or node in the name space) – Need to maintain mount information n Keep it at the clients n Keep it at the servers

15. 4 April 2001CS-551, Lecture 815 Name Space Hierarchy a i b c def gh jk Server X Server Z Server Y Adapted from Singhal & Shivaratri (1994)

16. 4 April 2001CS-551, Lecture 816 Mechanisms: Mounting, cont. n Keep it at the clients – Client must mount each required file system n e.g. Sun’s NFS – Each client can see a different filename space – When moving files, each client may need updating n Keep it at the servers – Each client sees identical filename space – If files are moved between servers, only need to update servers’ information

17. 4 April 2001CS-551, Lecture 817 Mechanisms, continued n Caching – Clients get copy of remote file information n Local memory, local disk, server memory – Improves performance n Hints – Guaranteeing that all data in cache is always valid is expensive – Some cached data can be used as a hint n If shown valid, then time is saved n If found invalid, can recover without serious problems – E.g. cache location of a file

18. 4 April 2001CS-551, Lecture 818 Mechanisms, concluded n Bulk Data Transfer – Big cost of communication is the communication protocol – So send multiple data blocks on each transfer n Less communication overhead n Less context switching n Fewer acknowledgements n Encryption – Enforce security – Before communication between two entities, use an authentication server to provide a key

19. 4 April 2001CS-551, Lecture 819 DFS Design Issues n Naming and Name Resolution n Caches on Disk or Main Memory n Writing Policy n Cache Consistency n Availability n Scalability n Semantics

20. 4 April 2001CS-551, Lecture 820 Naming and Name Resolution n Name Resolution – “The process of mapping a name to an object, or in the case of replication, multiple objects” (SS 94) n Name Space – “a collection of names which may or may not share an identical resolution mechanism” (SS 94)

21. 4 April 2001CS-551, Lecture 821 Name Space Hierarchy a i b c def gh jk Server X Server Z Server Y Adapted from Singhal & Shivaratri (1994)

22. 4 April 2001CS-551, Lecture 822 Figure 8.10 Name Space and Mounting in NFS.

23. 4 April 2001CS-551, Lecture 823 Naming Definitions n Location independent: A file can be moved without changing the filename n Location transparent: Filename does not tell where the file is located

24. 4 April 2001CS-551, Lecture 824 Location Transparency n Must be provided via global naming n Dependent on a name being location independent – E.g. a universal name n Example: social security number versus home street address

25. 4 April 2001CS-551, Lecture 825 Figure 8.1 Telephone Routing.

26. 4 April 2001CS-551, Lecture 826 Global Naming and Name Transparency n A global name space requires – Name resolution – Location resolution n Name resolution maps symbolic filenames to computer file names n Location resolution involves mapping global names to a location n Difficult if both name transparency and location transparency are both supported

27. 4 April 2001CS-551, Lecture 827 Figure 8.2 IP Hierarchical Name Space.

28. 4 April 2001CS-551, Lecture 828 Naming Approaches n Add host name to names of files on that host – Provides unique names – Loses network transparency – Loses location transparency – Moving file to a different host causes change of filename n Possible changes to applications using that file – Easy to find a file

29. 4 April 2001CS-551, Lecture 829 Naming Approaches, continued n Mount remote directories onto local directories – To do the mount, need to know host – Once mounted, references are location transparent – Can resolve filenames easily – However, a difficult approach to do n Not fault tolerant n File migration requires lots of updates

30. 4 April 2001CS-551, Lecture 830 Naming Approaches, concluded n Use a single global directory – Does not have disadvantages of previous approaches – Variations found in Sprite and Apollo – Need a single computing facility or a few with lots of cooperation n Need system-wide unique filenames – Not good on a heterogeneous system – Not good on a wide geographic system

31. 4 April 2001CS-551, Lecture 831 Naming Issues, continued n Contexts – Used to partition a name space n To avoid problems with system-wide unique names n Geographical, organizational, etc. – A name space in which to resolve a name – A filename has two parts n Context n Local filename – Almost like another level of directory – Used in x-Kernel logical file system

32. 4 April 2001CS-551, Lecture 832 Naming Issues, concluded n Name Server – Maps names to files and directories – Centralized n Easy to use n A bottleneck n Not fault tolerant – Distributed n Servers deal with different domains n Several servers may be needed to deal with all the components in a filename

33. 4 April 2001CS-551, Lecture 833 Figure 8.3 Distributed Solution for Name Resolution.

34. 4 April 2001CS-551, Lecture 834 DFS Design Issues, continued n File Cache Location – Main Memory n Can support diskless workstations n Faster n Similar to design of server memory cache n Competes with virtual memory system for space – Try to avoid data blocks being in both cache and virtual memory n Can’t cache a large file – So needs to be able to handle blocks (block-oriented)

35. 4 April 2001CS-551, Lecture 835 DFS Design Issues, continued n Cache Location, continued – Local Disk n Able to handle large files without affecting performance n Doesn’t affect virtual memory system n Permits incorporation of portable workstations into distributed system – As per Coda

36. 4 April 2001CS-551, Lecture 836 DFS Design Issues, continued n Cache Writing Policy – When should a modified cache block be sent to the server? – Write-through n Send all writes immediately to the servers n Reliable, little lost if there is a crash n Lose advantage of having a cache – Delayed writing

37. 4 April 2001CS-551, Lecture 837 DFS Design Issues, continued n Cache Writing Policy, continued – Delayed writing n Forward writes to server after a delay – E.g. when a block is full – E.g. when the file is closed – E.g. when timer goes off (say every 30 seconds) n Takes advantage of cache n Crash could lose some data – What about short-lived files (e.g. temps)? n Perhaps server need not know about these

38. 4 April 2001CS-551, Lecture 838 DFS Design Issues, continued n Cache Consistency – Server-Initiated n Server tells client that data needs to be updated – I.e. server needs good records n Client cache managers invalidate old data – Client-Initiated n Client cache manager makes sure client’s data is okay with server before using – Then why bother with cache at all? – Both these are expensive and require cooperation between clients and servers

39. 4 April 2001CS-551, Lecture 839 DFS Design Issues, continued n Cache Consistency, continued – Alternative n Do not allow file caching of shared, writeable files – As a concurrent-write sharing file may be open at multiple clients with at least one client writing n Server needs to keep track of clients sharing files n Can be avoided by locking files

40. 4 April 2001CS-551, Lecture 840 DFS Design Issues, continued n Cache Consistency, concluded – Issue: Sequential-write sharing n Occurs when a client opens a file that has been modified recently and closed by another client n Problem 1 – When client opens a file, it may have outdated blocks in its cache – Solution: use timestamps on files and cached blocks n Problem 2 – When client opens a file, current data blocks may still be waiting to be flushed in another client’s cache – Solution: Require all clients to flush modified file blocks when a new client opens file for writing

41. 4 April 2001CS-551, Lecture 841 Figure 8.7 Approaches to Modification Notification.

42. 4 April 2001CS-551, Lecture 842 DFS Design Issues, continued n Availability – Files can be unavailable due to server failures – Availability achieved through replication n Copies at different servers n Problems – Overhead (file space) – Consistency Need to maintain Need to detect and correct inconsistencies

43. 4 April 2001CS-551, Lecture 843 Availability, continued n Unit of replication – A file is the most common unit n Cedar, Roe, Sprite n Overall replica management is harder – Directory information about file may need to be stored (e.g. protection info) – Replicas of files belonging to a common directory may not have common file servers, requiring extra name resolutions

44. 4 April 2001CS-551, Lecture 844 Availability, continued n Unit of replication, continued – A group of files or Volume n Used by Coda n Easier to associate information with the group n A waste if most of the files are not really shared – Compromise n Used in Locus n A user’s files are a file group (primary pack) n A replica may just contain a subset of the pack

45. 4 April 2001CS-551, Lecture 845 Availability, concluded n Replication Management – Keeps mutual consistency among the copies – Suggest a weighted voting scheme n Reads/writes can happen only by votes from current copies n Timestamps are kept on current copies – Designate on or more processes as agents for controlling access to copies n Locus: each file group has a synchronization site n Harp: a primary file server controls access

46. 4 April 2001CS-551, Lecture 846 Figure 8.8 Employing a Mapping Table for Intermediate File Handles.

47. 4 April 2001CS-551, Lecture 847 Figure 8.9 Distributed File Replication Employing Group Communication.

48. 4 April 2001CS-551, Lecture 848 DFS Design Issues: Scalability n Can the design deal with system as it grows? n Caching is used to improve client response time n But it introduces cache consistency problems

49. 4 April 2001CS-551, Lecture 849 Scalability, continued n Server-initiated invalidation – Server keeps track of sharers n Notifies them if file is changed n Large system => busy server n Helps to note if file is read-only – Form a tree n Server only deals with only delta clients directly n Each of these clients can serve delta clients n Etc. – forming a tree for messages to propagate

50. 4 April 2001CS-551, Lecture 850 Scalability, continued n Server structure – Decides how many clients a server can support – Single process that blocks during the I/O n Horrible – all clients must wait – Separate process per client n Context switching overhead from frequent requests from different clients – Thread per client n Cheaper context switching

51. 4 April 2001CS-551, Lecture 851 Scalability, continued n Principle: – Minimize cross-machine interaction – Use caching, hints, relaxed sharing semantics n Stringent semantics are less scalable – Avoid central control and central resources n Central authentication service, name server, etc. – Desire symmetry and autonomy n Each machine has equal role – Decentralized system administration

52. 4 April 2001CS-551, Lecture 852 Scalability, concluded n Principle, concluded: – Clustering n Partition system into a collection of clusters – Cluster = set of machines plus cluster server n Hope most requests are satisfied by local cluster server – Balance and locality n With reasonable locality, clusters can be a scalable building block

53. 4 April 2001CS-551, Lecture 853 DFS Design Issues: Semantics n Characterizes the effects of accesses on files n Basic (Unix semantics) – A read operation returns the data stored by the last write operation – Expensive n Need a single coordinating server n OR no sharing n Or users need to use locks

54. 4 April 2001CS-551, Lecture 854 Semantics, concluded n Session semantics – Writes are visible immediately to local clients – Changes to a file are visible to remote clients, only after closing the file – No attempt to maintain consistency

55. 4 April 2001CS-551, Lecture 855 Distributed Directory Service n Directory Structures – Hierarchical – Acyclic n E.g., Unix – Cyclic n Directory Management – List of active directories with files – Storage of directory structure

56. 4 April 2001CS-551, Lecture 856 Directory Tree on one machine D Adapted from Tanenbaum (1995) E BC A

57. 4 April 2001CS-551, Lecture 857 Directory Graph on two machines D Adapted from Tanenbaum (1995) E BC A 1 0 11 2

58. 4 April 2001CS-551, Lecture 858 Distributed Directory Service n Directory Operations – Directory service n Create, rename, delete directories, etc. – File service n Create, rename, delete files, etc.

59. 4 April 2001CS-551, Lecture 859 File Types Library files (.lib,.dll) Program files (.c,.cpp,.p,.java,.f) Object-code files (.o,.obj) Compressed files (.zip,.Z,.gz) Archive files (.arc,.tar,.jar) Graphic files (.gif,.jpeg,.ps,.dvi) Sound files (.wav,.midi) Index files (.idx) Document files (.doc,.tex.,wp)

60. 4 April 2001CS-551, Lecture 860 Example DFSs n Sun NFS n Sprite n Apollo DOMAIN n X-Kernel n Coda n Andrew n Amoeba