Trends In Distributed File Systems Professor: Dr. Zhang CSc 8320 Advanced Operating Systems Presented by: Tu Tran Section 5.3

Outline New Hardware Scalability Wide Area Networking Mobile Users Fault Tolerance Multimedia Interesting Research Papers/Projects

New Hardware Costs continue to drop at an amazing rate Rapidly dropping memory costs make it possible to have every larger databases in main memory  Servers could have entire file system in main memory  Advantage: Gain in performance Simplify file system structure

New Hardware (cont’d) Simpler to store each file contiguously in memory (UNIX – tree, MS-DOS – link listed)  Disadvantage: Power fails, all files stored are lost. However, current technology allows to back up continuously files on tapes or other storage media. RAM disk and file system

New Hardware (cont’d) Optical Disk » Write-Once Read Many (WORM) » CD-ROM/DVD burners » Excellent backup and archiving method » Increasingly cheap, but slow. Huge capacity networks » Server with a main memory & a fast fiber optic network. May not need client’s cache and server’s disk. Files are backed up by optical disk.

New Hardware (cont’d) » Problem: memory cache in multiprocessor is similar to the case in which two clients are caching the same file. » Solution: design a simple network interface which has a bit map, one bit per cached file. Specialized hardware for sophisticated systems » Real-time support » Distributed synchronization and control

Scalability Distributed system size strongly affects algorithm choice » Working well for 100 machines means nothing for 10,000. Centralized algorithms do not scale well » Opening a file requires to contact the centralized server to record the fact that the file is open. As the system grows, the algorithms do not work well.

Scalability (cont’d) » Solution: partitioning the system into smaller units often helps Broadcasts are a problem » Consider CPU broadcasting one message per second » N of these generate N interrupts at N machines » Not a problem for N=10 » VERY problematic for N=10,000

Scalability (cont’d) Data structures become important with scaling » Linear search easiest and fastest for 10 » Self abuse for even 100 Strict semantics (Unix) are harder to implement as systems scale » Design Principle: use weakest semantics that make sense

Scalability (cont’d) » Trade off ease of programming with scalability Name space » A single Unix-like file tree. How long can/should path names get? » Partition the tree into smaller trees.

Wide Area Networking Virtually all distributed system research has been done in the context of LANs » Considerable changes with WAN context » Latency » Loss » Cost » Interaction

Wide Area Networking (cont’d) WAN access of major economic importance » WWW commerce » Video on demand » Distributed Virtual Environments Economies of scale » IP phones

Mobile Users Network addressing is a big challenge - Mobile IP » May be transparent to distributed computing level Often seen as highly variable communication bandwidth » Isolated » Wireless » Wired

Mobile Users (cont’d) Interesting effects on caching » CODA file system claims to support mobility and intermittent connection Rapidly Deployable Radio Network – RDRN [2] [2] » Wireless end-user and network nodes » Steerable communication beams » Self-organizing network structure

Fault Tolerance Most systems are not fault tolerant » But the general population expects things to work » Phone system  IP phones? Requires considerable redundancy » Hardware » Communication infrastructure » Software » Data – File replication

Fault Tolerance (cont’d) Downtimes and periodic crashes will become less and less acceptable as computers spread to non-specialists » ATM machines » Microwaves » Phone system (IP mode)

Multimedia Current data files are rarely more than a few megabytes » Multimedia files can exceed gigabytes » Compression clearly popular because of this and has a fundamental affect on network requirements and economics

Multimedia (cont’d) Video-on-demand » Significant affect on network traffic » Perhaps also on file systems » Real-time support is interesting as well

Interesting Research Papers/Projects xFS: Serverless Network File Service [3] [3] Distributed file systems for grid computing [4] [4] Mobile and Distributed file systems [5,6,7] [5,6,7] A Scalable Distributed File System [8] [8]

References [1] Andrew S. Tanenbaum. Distributed Operating Systems. Prentice-Hall, Englewood Cliffs, NJ 07632, USA, 1995 [2] Rapidly Deployable Radio Network, [3] xFS: Serverless Network File Service, [4]

References (cont’d) [5] MFS: an Adaptive Distributed File System for Mobile Hosts [6] Azzedine Boukerche, Raed Al-Shaikh, Bo Marleau, "Disconnection- Resilient File System for Mobile Clients," lcn, pp , The IEEE Conference on Local Computer Networks 30th Anniversary (LCN'05)l, er= &isnumber= er= &isnumber=33047 [7] Haddock-FS, [8] DiFFS: a Scalable Distributed File System