Cooperative Caching, Simplified Austin Chen

What is Cooperative Caching? Method for data read performance optimization on client-server architecture Builds off the fast speed of cache on local machines Combines the speed of individual local client caches to boost client read speeds across entire system

Quick Review of Memory Hierarchy

Quick Review of Memory Hierarchy Disk reads can be time consuming When one client needs to read information from another client’s disk, it is slow Hint: there are ways to improve this

How it might work in a simple system Non-cooperative caching systems might use a three-level memory hierarchy - Local Memory - Server Memory - Server Disk

The Cooperative Caching Method Cooperative caching introduces a different client’s memory - Local Memory - Server Memory - Server Disk - Another client’s local memory

How is this all possible? Processor speed has improved faster than disk performance Requires a high-speed, low-latency network that surpasses the speed of hard disk reads Fetching data from remote memory is ~3 times faster than getting data from a remote disk. With cooperative caching, remote memory can be accessed ten to twenty times as quickly as disk.

Four Cooperative Caching Algorithms Direct Client Cooperation Greedy Forwarding Algorithm Centrally Coordinated Caching N-Chance Forwarding

Direct Client Cooperation Simplest algorithm Once the local cache of a client fills and overflows, it forwards cache entries onto an idle machine’s cache The active client can now access the idle machine’s cache to satisfy read requests

Direct Client Cooperation

Direct Client Cooperation Pros: Simple, can be implemented without overall server modification Cons: The client donating its cache MUST be idle. As far as the server is concerned, a client utilizing remote memory appears to have a temporarily enlarged cache. Con: If all clients are active, then no one computer can leverage cooperative caching.

Greedy Forwarding Algorithm Each client manages it’s own cache “greedily”, meaning it does not have permissions to modify another client’s cache. If the client does not find a block in its local cache, it looks in the server cache. If not in server cache, then looks at client caches.

Greedy Forwarding Algorithm Needs this block of data:

Greedy Forwarding Algorithm Pros: Appealing because it’s “fair”. Clients need only worry about managing local resources. Cons: Lack of data coordination could lead to data duplication

Centrally Coordinated Caching Attempts to lessen data duplication problem through coordinated caches Each client cache is statically partitioned into a locally managed section and a globally managed section by the server

Centrally Coordinated Caching = Globally Managed Cache

Local vs. Global cache… how does it work? Server manages the globally managed portion with a “global replacement algorithm” LRU Cache When the server evicts a block from its local cache to make room for data, it sends the evicted block to the LRU cache the global cache. The global cache will then boot off the oldest, least used block and to make room for the overflow.

Centrally Coordinated Caching Pros: High global hit rate it can achieve through global management Cons: Each individual’s client has a reduced cache size. Local hit rate is reduced. Pros: Because it manages a global cache, it can cover a lot of ground and not create duplicate entries. Cons: Another drawback is that centrally coordinating a cache may impose significant load on the server.

N-Chance Forwarding Each client cache is now dynamically partitioned based on client activity N-Chance algorithm has the individual clients preferentially cache singlets Singlets are blocks only stored in one client’s cache

= Globally Managed Cache N-Chance Forwarding = Globally Managed Cache

N-Chance Forwarding Pros: Optimizes central coordinated caching by taking into account client activity Cons: Can be prone to bouncing a block of data among multiple caches if the global cache is being resized. Extra server load.

Performance, compared

Why not just get more server RAM? Server is less loaded overall because it handles network requests instead of large data disk lookups Cooperative cache systems are more cost effective It’s cheaper to add 16GB of RAM to 100 clients than add one big 1.6TB chunk of RAM to a single server https://downloadmoreram.com/