1. Distributed Systems CS 15-440
Caching – Part II
Lecture 16, November 6, 2017
Mohammad Hammoud

2. Today… Last Lecture: Today’s Lecture: Announcements:
GraphLab & Intro to Caching
Today’s Lecture:
Continue with Caching
Announcements:
P3 is due on Nov 12th by midnight
Quiz II is on Nov 16 (during the recitation time)

3. Three Key Questions What data should be cached and when?
Fetching Policy
How can updates be made visible everywhere?
Consistency or Update Propagation Policy
What data should be evicted to free up space?
Cache Replacement Policy

4. Three Key Questions What data should be cached and when?
Fetching Policy
How can updates be made visible everywhere?
Consistency or Update Propagation Policy
What data should be evicted to free up space?
Cache Replacement Policy

5. Fetching Policy Two broad types: Push-based fetching policy
Pull-based fetching policy

6. Push-Based Caching Push-based Caching (or Full Replication)
Every participating machine gets a complete copy of data in advance
Every new file gets pushed to all participating machines
Every update on a file is pushed immediately to every corresponding replica
Example: Dropbox
Works well enough in practice (technical excellence is only weakly correlated to business success)

7. Push-Based Caching: Scalability Issues
Clearly, this can create a major scalability issue
With larger team sizes and/or datasets, the push-based model consumes larger amounts of network bandwidth and disk spaces
At very large-scale, it might take a day to finish a sync operation!
This defeats the very purpose of full replication, which is usually to enable collaboration among teams
A different approach referred to as pull-based caching attempts to solve this problem

8. Pull-Based Caching Pull-based Caching (or On-demand Caching)
A file is fetched only if needed
Updates on a file (not necessarily the whole file) are propagated to replicated files only if needed
This leads to a more fine-grained and selective approach (as opposed to the push-based model) for data management
Examples: AFS

9. Three Key Questions What data should be cached and when?
Fetching Policy
How can updates be made visible everywhere?
Consistency or Update Propagation Policy
What data should be evicted to free up space?
Cache Replacement Policy

10. One-Copy Semantic
Caching Reality
Desired Illusion
One-Copy Semantic

11. One-Copy Semantic
A caching system has one-copy semantic if and only if:
There are no externally observable functional differences with respect to an equivalent system that does no caching
However, performance/timing differences may be visible
This is very difficult to achieve in practice
Except in very narrow circumstances like HPC-oriented file systems and DSMs
All real implementations are approximations

12. What Makes One-Copy Semantic Hard to Implement?
Physical master copy may not exist
Nodes usually track who has the most recent copy
More likely to occur in peer-to-peer than in master-slave systems
Network may break between some clients and the master copy
Disconnected sites see no further updates
Sites may split into regions, which might get isolated from each others
Low read-to-write ratios
This generates huge amount of cache propagation traffic
The interconnect might become the bottleneck for cache accesses
Neither writer-bias (write-back) nor reader-bias (write-through) helps
Caching might render effectively useless

13. Cache Consistency Approaches
We will study 7 cache consistency approaches:
Broadcast Invalidations
Check on Use
Callback
Leases
Skip Scary Parts
Faith-Based Caching
Pass the Buck

14. Cache Consistency Approaches
We will study 7 cache consistency approaches:
Broadcast Invalidations
Check on Use
Callback
Leases
Skip Scary Parts
Faith-Based Caching
Pass the Buck

15. Broadcast Invalidations
A write goes as follows:
Reads on cached objects can proceed directly
(F1, F2, F3)
Server
Go Ahead
Need to Write on F1
Write-
back F1
Invalidate F1
(F1)
Client 1
Write on F1
Ack
(F2)
(F1, F2)
Client 2
Negative-Ack
(F3)
Client 3

16. Broadcast Invalidations
The server does not maintain a directory that keeps track of who is currently caching every object
Thus, upon any update to any object, the server broadcasts an invalidation message to every caching site
If a site is caching the object, it invalidates it; otherwise, it sends a negative Ack message to the server
If invalidated, next reference to this object at this site will cause a miss

17. Broadcast Invalidations
Advantages:
No special state (except locations of caching sites) is maintained at the server
A stateless server
No race conditions can occur if an updater blocks until all the cached copies of the requested object (except its own) are invalidated
Very strict emulation of the one-copy semantic
Simple to implement

18. Broadcast Invalidations
Disadvantages:
Traffic is wasted, especially if no site caches the requested object
The updater blocks until the invalidation process is completed
Not scalable in large networks
Could lead to flooding the network if the number of writes is high and the read/write ratio is low
Requires that all sites listen (or snoop) to all requests

19. Cache Consistency Approaches
We will study 7 cache consistency approaches:
Broadcast Invalidations
Check on Use
Callback
Leases
Skip Scary Parts
Faith-Based Caching
Pass the Buck

20. Check on Use The server does not invalidate cached copies upon updates
Rather, a requestor at any site checks with the server before using any object
Versioning can be used, wherein each copy of a file is given a version number
Is my copy still valid?
If no, fetch a new copy of the object
If yes and I am a reader, proceed
If yes and I am a writer, proceed and write-back when done

21. Check on Use
Has to be done at coarse-granularity (e.g., entire file or large blocks)
Otherwise, reads are slowed down excessively
It results in session semantic if done at whole file granularity
Open {Read | Write}* Close  “session”
Updates on an open file are initially visible only to the updater of the file
Only when the file is closed, the changes are made visible to the server

22. Check on Use Disadvantages: Concurrent Updates!
“Up-to-date” is relative to network latency
(F)
Server
Is version of file F
still X?
YES
YES
Write-Back
(F)
Client 1
Update
Is version of file F
still X?
Write-Back
(F)
Client 2
Update
Concurrent Updates!

23. Check on Use Disadvantages: How to handle concurrent writes?
The final result depends on whose write-back arrives last at the server
This gets impacted by network latency
If updates A and B are exactly the same
And the machines where they are pursued are homogenous
And A is started, finished, and sent before B
It is not necessary that A will reach the server before B
Slow reads
Especially with loaded servers and high-latency networks

24. Check on Use Disadvantages: Advantages:
Pessimistic approach, especially with read-most workloads
Can we employ an optimistic (or Trust-and-Verify) approach?
Advantages:
Strict consistency (not across all copies) at coarse granularity
No special server state is needed
Servers do not need to know anything about caching sites
Easy to implement

25. Cache Consistency Approaches
We will study 7 cache consistency approaches:
Broadcast Invalidations
Check on Use
Callback
Leases
Skip Scary Parts
Faith-Based Caching
Pass the Buck

26. Callback A write goes as follows:
Reads on cached objects can proceed directly
(F1, F2, F3)
Server Go
Ahead
Write-
back F1
Need to Write on F1
Invalidate F1
(F1)
Client 1
Write on F1
Ack
(F2)
(F1, F2)
Client 2
(F3)
Client 3

27. Callback
The server maintains a directory that keeps track of who is currently caching every object
Thus, upon an update to an object, the server sends invalidation messages (i.e., or callbacks) only to sites that are currently caching the object
Typically done at coarse granularity (e.g., entire file)
Can be made to work with byte ranges

28. Callback Advantages: Targeted notification of caching sites
Zero network traffic for reads of cached objects
Biases read performance in favor of write- performance
Excellent scalability, especially with read-most workloads

29. Callback Disadvantages:
Complexity of tracking cached objects on clients
Sizable state on server
Silence at the server is ambiguous for clients
What if a client has been reading a file for a little while without hearing back from the server?
Perhaps the server is down
A keep-alive (or heartbeat) mechanism can be incorporated, whereby the server pings the clients (or the other way around) every now and then indicating that he is still alive

30. Cache Consistency Approaches
We will study 7 cache consistency approaches:
Broadcast Invalidations
Check on Use
Callback
Leases
Skip Scary Parts
Faith-Based Caching
Pass the Buck

31. Leases
A requestor needs to obtain a finite-duration control from the server
This duration is called a lease period (typically, for few seconds)
There are three types of leases
Read and write leases, assuming an invalidation-based protocol
Multiple requestors can obtain read leases on the same object, but only one can get a write lease on any object
Open leases, assuming a check-on-use protocol
A requestor loses control when its lease expires
If needed, the requestor can renew the lease

32. Synchronized Clocks are Assumed at All Sites
Lease Renewal
Example:
(F)
Server
Sorry, I can give
you a read lease
for time Y on F
Renew my read lease for another time Y
on F
Okay, you got
an extension for
Y time on your
read lease over F
Give me a read
lease for time X on F
(F)
Client 1
Read F for duration Y
Read F for duration Y
Synchronized Clocks are Assumed at All Sites

33. Next Class
Continue on cache consistency approaches