1. CODA/AFS (Constant Data Availability)
--Jay N. Vyas & Naveen Mohan--

2. Background CODA is what? – A Distributed File System (DFS)
Developed by? – Mahadev Satyanarayanan
Where & Why? – At Carnegie Mellon University as a Research Project in 1987

3. CODA/AFS - Introduction
A decentralized distributed file system (DFS) to be accessed from autonomous workstations.
Derived from AFS (Andrew File System) and inherits many of the core features.
AFS was a very successful DFS for a campus-sized user community.
Was planned to be extended at a national level but WWW took over instead.
Key ideas of AFS include:
Close-to-open consistency
Callbacks

4. Design Motto CODA had three fundamental objectives:
Scalability - to build a system that can easily cope with addition of users and sites
Fault-Tolerance - to remain usable in the advent of server failures, communication failures and voluntary disconnections
Unix Emulation – to make Coda appear as a giant failure-proof shared Unix file system

5. Key Features How is CODA different from AFS then? - Fault Tolerance!
Provides “constant availability” through Replication
Introduces a “disconnected mode” for portable computers (Mobile Computing)
Supplementary Features:
Is freely available under a Liberal License (Source Code is accessible)
High Performance through Client side persistent caching
Uninterrupted service during partial network failures
Highly Scalable
Provides Unified file sharing using well defined semantics even in the event of network failures

6. History: AFS-1 semantics
First version of AFS:
Revalidated cached file on each open
Propagated modified files when they were closed
If two users on two different workstations modify the same file at the same time, the users closing the file last will overwrite the changes made by the other user
A successful open implied that the resulting copy of the file was the latest in the system

7. Open-to-Close Semantics
Example:
Time F F’ F”
First client
Second client
F” overwrites F’
First client F’
F” overwrites F’ F’ F” F
Time
Second client

8. AFS-2 semantics
AFS-1 required each client to call the server every time it was opening an AFS file
Most of these calls were unnecessary as user files are rarely shared
AFS-2 introduces the callback mechanism
Do not call the server, it will call you!
When a client opens an AFS file for the first time, server promises to notify it whenever it receives a new version of the file from any other client
Promise is called a callback
Relieves the server from having to answer a call from the client every time the file is opened
Significant reduction of server workload

9. AFS-2 semantics (Contd..)
Callbacks can be lost!
Client will call the server every tau minutes to check whether it received all callbacks it should have received
Cached copy is only guaranteed to reflect the state of the server copy up to tau minutes before the time the client opened the file for the last time

10. Coda semantics
Client keeps track of subset s of servers it was able to connect the last time it tried
Updates s at least every tau seconds
At open time, client checks it has the most recent copy of file among all servers in s
Cached copy can be up to tau minutes behind the server copy
Conflicts between cached copy and most recent copy in s can happen
Coda guarantees they will always be detected
At close time, client propagates the new version of the file to all servers in s
If s is empty. client marks the file for later propagation
When s is empty, it is said to operate in disconnected mode.
- A successful close indicates that the file has been propagated to the set of currently accessible servers, or
that no server is available and the file has been marked for propagation at the earliest opportunity.

11. Server Replication Data structures: Volume
A set of files and directories located on one server (shared)
Each file and directory in Coda has a unique low-level file identifier (FID). All replicas of an object has the same FID.
VSG (Volume Storage Group):
Set of servers storing replicas of a volume.
AVSG (Accessible Volume Storage Group):
Venus (client cache manager) keeps track of the subset of servers accessible to the client for every volume the client has cached.
Preferred Server:
One of the AVSG servers of the client, that it accesses during a cache miss. Chosen based on physical proximity, server load or server CPU power.
Holds all callbacks for client & answers all the read – write requests from client.

12. Server Replication(contd..)
Strategy
Read – any , Write – all approach
Client still checks with other servers (in AVSG) to find which one has the latest version/copy of the data.
When a file is closed after modification by a client it is transferred in parallel to all members of the AVSG.
Minimizes Server CPU load
Improved scalability, since Server is the bottleneck in many distributed file systems.
Cache Coherence – Client identifies 3 types of events no later than Τ seconds,
Enlargement of AVSG – Accessibility of a previously inaccessible server.
Shrinking of AVSG – Inaccessibility of a previously accessible server.
Lost Callback event
Coda Version Vector ( CVV )
A volume CVV is similar to a file or directory CVV, but summarizes update information on the entire volume.
- Server CPU load is minimized because the burden of data propagation is on the client rather than the server.
At present, a server performs no explicit remote actions upon recovery from a crash. Rather, it depends upon clients to notify it of stale or conflicting data.
- CVV : It is updated as a side-effect of every operation that modifies the volume. A mismatch in the volume CVV’s indicates that some AVSG members missed an update

13. Parallel Communication
Replication requires multiple sites to be contacted.
If serialized, latency would be degraded intolerably.
MultiRPC - parallel remote procedure call mechanism which were previously used was extended to use hardware multicast to reduce the latency and network load.

14. Server Replication (Conflict Scenario)
Unit of replication: volume
Volume Storage Group (VSG): set of servers that have a copy of a volume
Accessible Volume Storage Group (AVSG): set of servers in VSG that the client can contact
Use vector versioning
One entry for each server in VSG
When file updated, corresponding version in AVSG is updated
Versioning vector (Coda Version Vector) when partition happens: [1,1,1]
Client A updates file  versioning vector in its partition: [2,2,1]
Client B updates file  versioning vector in its partition: [1,1,2]
Partition repaired  compare versioning vectors: conflict!

15. Conflict Resolution Conflict Detection
Coda does not prevent inconsistent updates but guarantees that inconsistent updates will always be detected
Automated Resolution
All directory conflicts are automatically resolved by a compensating sequence of create or delete operations, except
Update/update conflict
Remove/update conflict
Name/name conflict
Damage Containment
Marks all accessible replicas of the objects inconsistent. (if can’t be automatically resolved).
Repair Tool (Manual)
Special Interface to Venus (Client Cache Manger)
Enables the tool to overwrite inconsistent files and to perform directory operations on inconsistent directories, subject to normal access restrictions
The Coda repair tool allows users to manually resolve conflicts.
It uses a special interface to Venus so that file requests
from the tool are distinguishable from normal file requests.

16. Server Replication - Performance

17. Introducing Mounting Operation
The Andrew file system, Coda's predecessor, pioneered the idea and stored all files under /afs.
A client (user) connects to “Coda” and not to individual servers; but the latter come into play invisibly.
All the files, which any of the servers may provide to the client, are available under the /coda directory, and all clients see the same name space.
Why is Single mount point (name space) advantageous:
Users will see the same file tree
Auto-discovery of new servers and shares by the Client machines in /coda tree

18. CODA Operation – Disconnection Mode
How does it provide services when the Network crashes?
Scenario:
User wants to open a file “foo” – cat /coda/tmp/foo
A System call is generated to the kernel and it looks for inode of the file which returns the file handle associated with that program (what is inode? RECALL!)
Open call proceeds to the Virtual File System (VFS) (What is VFS?) which is handed to the Coda File system present on the Kernel
CFS in turn “may” forward the request to “Venus” (What is Venus? RECALL!)
Venus checks first in the client cache for /tmp/foo, if cache-miss, then contacts the Vice servers.
After successful file location, Venus responds back to the kernel with info which returns to the calling program.

19. Fig. The Architecture

20. When does Disconnection Mode crank up?
Possibilities:
When there is no network connection to any server which has the files.
Happens for laptops when taken off the network, or during network failures.
Or if the servers fail.

21. And how does it work? (Caching)
Entire file fetched from the servers and stored as a container file in the local cache area by Venus on receiving the first kernel request
(currently/usr/coda/venus.cache/).
If the file is opened a second time or R/W operations performed, local copy is only used.
If it’s modified and closed, then Venus updates the servers by sending the new file.
Operations like making directories, removing files or directories and creating or removing (symbolic) links are propagated to the servers also.

22. From Caching to Disconnection
Servers synchronously updated with modified files in normal connection mode
Time-out occurs if Server unavailable or Network failure
The Ball Begins to Roll!!! (Activate Disconnect Mode)
Instead of instantly updating the user with time-out, Venus logs all updates in CML (Client Modification Log); frequently flushed to the disk
Upon reconnection of the network, CML file is reintegrated with the server– Run Updates
Another Fault Tolerance feature achieved with crisp Transparency…

23. Issues with disconnected operation
Hoarding Files
Unable to serve cache-misses during network anomaly
Hence Stack up updates to the important files on local cache (client side)
Known as “Hoard Walk”
Client/Server Conflict
Multiple clients may have modified the file during disconnect mode to CML.
Needs Repair – either Automatically (Resolvers) with rare chances of Human intervention

24. CONCLUSION & Paper Highlights
Coda project had an unusually long life
First work to introduce disconnected mode
Server Replication resulting in increased System resiliency
Coda/AFS introduced many new ideas
Close-to-open consistency
Optimistic concurrency control
Callbacks (partially superseded by leases)

25. References Coda File System: http://www.coda.cs.cmu.edu
"The Coda Distributed File System." The Coda Distributed File System. N.p., n.d. Web. 11 Mar URL:
Mahadev Satyanarayanan; Kistler, J.J.; Kumar, P.; Okasaki, M.E.; Siegel, E.H.; Steere, D.C., "Coda: a highly available file system for a distributed workstation environment," Computers, IEEE Transactions on, vol.39, no.4, pp.447,459, Apr doi: / URL: