Dr. Yingwu Zhu Summary Cache : A Scalable Wide- Area Web Cache Sharing Protocol

Contents Background and Problems Cache Cooperation ICP Summary Cache  Bloom Filters – the math  Bloom Filters as Summaries Evaluation Conclusions

Web Caching Picture Proxy Caches Users Regional Network Rest of Internet Bottleneck...

What we have and What to be done ? A single proxy cache serves a population  Many such proxy caches out there!  Can they cooperate as a single “large cache” to serve the aggregated population to increase hit ratio, reduce latency and bandwidth cost?  Proxy caches should cooperate and serve each other’s misses If so, how do we know which caches hold the requested objects that miss in current proxy cache?  Content location among cooperating caches?

ICP – What we have! Inter Cache Protocol (ICP)  First proposed in the context of the Harvest project  Supports discovery and retrieval of documents from neighboring caches

Cache Cooperation via ICP  When one proxy has a cache miss, send queries to all siblings (and parents): “do you have the URL?”  Whoever responds first with “Yes”, send a request to fetch the file  If no “Yes” response within certain time limit, send request to Web server Parent Cache (optional)

Cache Cooperation Types No cache sharing  Proxies do not collaborate Simple cache sharing (ICP-style)  Proxy caches the document locally from other proxies  Proxies do not coordinate cache replacement (each LRU) Single-copy cache sharing (global LRU)  Proxy not cache documents fetched from others  Proxy marks documents as most-recently-access Global cache  Proxy fully coordinate in servicing misses and cache replacement

What are the benefits of cache cooperation? Let Experiments showcase the benefits. Data traces used.

Traces and Simulations (1/2) Use five sets of traces of HTTP requests  Digital Equipment Corporation Web Proxy Server Traces (DEC)  Traces of HTTP request from the University of California at Berkeley Dial-IP service (UCB)  Traces of HTTP request made by users in the Computer Science Department, University of Pisa, Italy (UPisa)  Logs of HTTP GET requests seen by the parent proxies at Questnet, a regional network in Australia (Questnet)  One-day log of HTTP request to the four major parent proxies, “bo”, “pb”, “sd”, and “uc” in the National Web Cache gierarchy by National Lab of Applied Network Research (NLANR)

Traces and Simulations (2/2) Statistics about the traces  The maximum cache hit ratio and byte hit ratio are achieved with the infinite cache  Infinite cache size - Total size in bytes of unique documents in a trace  The other hit ratios are calculated assuming a cache size that is 10% of the infinite cache  All use LRU as the cache replacement algorithm  Restriction that documents larger than 250KB are not cached  Assuming that each group has its own proxy (splitting traces into groups)  Simulated the cache sharing among the proxies

Benefits of Cache Cooperation

What we got so far? Cache cooperation is good, even for simple cache cooperation! So we need collaboration among proxy caches! Now, we really need to address the issue  How can a proxy know its misses can be served by other cooperative proxies?  We know ICP can do that.  So turn our attention to ICP again!

ICP: Good but … ICP discovers cache hits in other proxies by having the proxy multicast a query message  Thus, as the number of proxies increases, both the communication and the processing overhead increase quadratically  T = N * (N-1)*(1-H)*R  T: Total ICP message  N: Number of Proxies  H: Hit rate  R: Average requests

Overhead of ICP Though the Internet Cache Protocol (ICP) has been successful at encouraging Web cache sharing around the world, it is not a scalable protocol  It relies on multicasting query messages to find remote cache hits  Every time one proxy has a cache miss, everyone else receives and processes a query message  As the number of collaborating proxies increases, the overhead quickly becomes prohibitive

Overhead of ICP Overhead of ICP in the four-proxy case  Environments  10 Sun Sparc-20 workstations connected with 100Mbps Ethernet  Four workstations act as four proxy systems running Squid , and each has 75MB of cache space  Four workstations run 120 client processes, 30 processes on each workstations  The client processes on each workstation connect to one of the proxies  Document sizes follow the Pareto distribution with α=1.1 and k=3.0  Two workstations act as servers, each with 15 servers listening on different ports

ICP: Overhead! Quantify the overhead of the ICP protocol, the number of proxies is 4  ICP increases the interproxy traffic by a factor of 70 to 90  CPU overhead by over 15%  Average user latency by up to 11%

Overhead of ICP The result highlight the dilemma faced by cache administrators  There are clear benefits of cache cooperation  But the overhead of ICP is high To address the problem, the authors proposed a new scalable protocol: “Summary Cache”

Alternatives to ICP Force all users to go through the same cache or the same ar ray of caches  Partition URLs among cooperating caches  Difficult in a wide-area environment Central directory server  Directory server can be a bottleneck Ideally, one wants a protocol: keeps the total cache hit ratio high minimizes inter-proxy traffic scales to a large number of proxies

Summary Cache Basic idea:  Let each proxy keep a directory of what URLs are cached in every other proxy, and use the directory as a filter to reduce number of queries Problem 1: keeping the directory up-to-date  Solution: delay and batch the updates => directory can be slightly out-of-date (trade freshness for #-of- msgs reducing) Problem 2: DRAM requirement  Solution: compress the directory => imprecise, but inclusive directory

Summary Cache Proxy a compact summary of the cache directory of other proxies  That is, the list of URLs of cached documents Only send query message to correct proxy When a cache miss occurs, a proxy first probes all the summaries The summaries do not need to be accurate at all times  A false hit, the penalty is a wasted query message  A false miss, the penalty is a higher miss ratio Two key questions in the design of the protocol  Frequency of summary updates  Representation of summary

Summary Cache Scalability of summaries  The key to the scalability of the scheme is that summaries do not have to be up-to-date or accurate  Summary does not have to be updated every time the cache directory is changed  The update can occur upon regular time intervals  Small size, which does not consume much memory

Summary Cache Two kinds of errors are tolerated  False misses  The document requested is cached at some other proxy but its summary does not reflect the fact  In this case  A remote cache hit is lost  The total hit ratio within the collection of caches is reduced  False hits  The documents requested is cached at some other proxy but its summary indicates that it is  The proxy will send a query message to the other proxy, only to be notified that the document is not cached there  In this case  A query message is wasted

Summary Cache A third kind of error, remote stale hits  Occurs in both summary cache and ICP  When a document is cached at another proxy, but the cached copy is stale Two factors limit the scalability of summary cache  Network overhead  Inter-proxy traffic  Memory required to store the summaries  For performance reasons, the summaries should be stored in DRAM, not on disk

Impact of Update Delays  ICP : No update delay  exact_dir : update delay increases (delay the updates until a certain percentage of the cached documents are “new”)

Summary Representation In practice, proxies typically have 8GB to 20GB of cache space If we assume 16 proxies of 8GB cache and file size of 8KB (16-byte MD5 per URL)  Exact-directory summary would consume = (16-1)*16*(8GB/8KB) = 240MB per proxy The requirement on an ideal summary representation  Small size, Inclusive, and low false hit ratio  Solution, “Bloom Filters”

Alternatives vs. Bloom Filters First try: use server URLs only  Problem: too many false hits, leading to too many messages between proxies Exact Directory  Problem: memory consuming, message size  Impractical!!! Bloom Filters:  Have the best of both!

Bloom Filters – the math If any of them is 0, then certainly b is not in the set A Otherwise we conjecture that b is in the set although there is a certain probability that we are wrong  False Positive (=> False Hit)  Clear tradeoff between m/n and the probability of a false positive

Bloom Filters: the Math Given n keys, how to choose m and k? Bit Vector v m bits Suppose m is fixed (>2n), choose k: k is optimal when exactly half of the bits are 0 => optimal k = ln(2) * m/n False positive ratio under optimal k is (1/2) k => false positive ratio = (1/2) ln2 * m/n = (0.62) m/n

Bloom Filters: the Practice Choosing hash functions  bits from MD5 signatures of URLs Maintaining the summary  the proxy maintains an array of counters  for each bit, the counter records how many times the bit is set to 1 Updating the summary  either the whole bit array or the positions of change d bits (delta encoding)

Bloom Filters as Summaries Total hit ratio under different summary representations m/n = 8, 16, 32

Bloom Filters as Summaries Ratio of false hits under different summary representations

Bloom Filters as Summaries Storage requirement (relative to infinite cache size)  In terms of percentage of proxy cache size, of the summary representations

Bloom Filters as Summaries Number of network messages per user request under different summary forms

Bloom Filters as Summaries Bytes of network messages per user request under different summary forms

Enhancing ICP with Summary Cache Prototype implemented in Squid Repeating the 4-proxy experiments, the new ICP:  Reduces UDP messages by a factor of 12 to 50 compared with the old ICP  Little increase in network packets over no cache sharing  increase CPU time by 2 - 7%  reduce user latency up to 4% with remote cache hits

Conclusion Summary Cache enhanced ICP Using trace-driven simulations and measurements Reduces the number of interproxy protocol message by factor of 25 to 60 Reduces the bandwidth consumption by over 50% Almost no degradation in the cache hit ratios Reduces CPU overhead between 30% to 95%

Questions