Announcements Now: focus on your Midterm 1, one week from today. Now: focus on your Midterm 1, one week from today.  Make sure you understand the papers. This Thursday is your turn to ask questions. One week from today is my turn.  7 Papers for Midterm 1: Beyond Rel DBs, RAID, Flash, Gamma, Bigtable, MapReduce, R-trees. March 10th: MIDTERM 1 March 10th: MIDTERM 1 March 5th: Perliminary draft of the project will be available. March 5th: Perliminary draft of the project will be available. March 26th: Document Design detailing how you will implement the algorithm. March 26th: Document Design detailing how you will implement the algorithm. April 9th: First implementation based on simple tests. April 9th: First implementation based on simple tests. April 16th: Second implementation based on traces. April 16th: Second implementation based on traces. April 23rd: A competition to chose the fastest implementation. April 23rd: A competition to chose the fastest implementation. April 30th: Midterm 2 April 30th: Midterm 2

Greedy Cache Management Techniques for Mobile Devices. by S. Ghandeharizadeh and S. Shayandeh.

Outline Target environment: A roaming wireless device Target environment: A roaming wireless device Cache management strategies: Cache management strategies:  Offline: Simple  Online: LRU-K, GreedyDual  LRU-K does not support variable sized clips  GreedyDual does not support fix sized clips/frames Improved variations: Improved variations:  Dynamic Simple, LRU-SK, IGD Conclusions and future research Conclusions and future research

Target Environment Today’s cell phone: Today’s cell phone:  Camera  Camcorder  Music jukebox  Color video display  Web browser  Voice services A device is equipped with mass storage and may cache clips: A device is equipped with mass storage and may cache clips:  Minimize the load on the wireless network,  Download a clip faster and before the wireless connection is lost,  Enable users to display cache resident clips when a device is not in the radio range of a base station, e.g., in an airplane. Example device: Example device: Apple’s iPhone

FMC Phones Fixed-Mobile-Convergence (FMC) phones equipped with both cellular and Wi-Fi networking cards Fixed-Mobile-Convergence (FMC) phones equipped with both cellular and Wi-Fi networking cards Wi-Fi provides limited radio range and high bandwidth. Wi-Fi provides limited radio range and high bandwidth. FMC phone use the Wi-Fi to detect base stations and roam onto VoIP. Why? FMC phone use the Wi-Fi to detect base stations and roam onto VoIP. Why? 1. Cheaper for customer 2. Better quality of service (higher bandwidth) 3. One number everywhere – on the road, at work, at home, when visiting a friend, etc. 4. Reliability

Reliability

Cache Management Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache. Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache.  Based on the principle of reference locality: the user repeatedly references only a limited amount of data that is available out there.  Byte hit rate: Percentage of data that is serviced using local cache. A 90% byte hit rate means 90% of retrieved data came from the cache. Why? Why?  ↑ cache hit rate means ↑ availability of data when a device has no base station coverage. How to manage cache? How to manage cache?  Offline techniques: Simple  Assumes advance knowledge of requests: Both “When?” and “what?”  Online techniques: LRU-K, GreedyDual What does “byte hit rate” optimize for?

Cache Management Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache. Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache.  Based on the principle of reference locality: the user repeatedly references only a limited amount of data that is available out there.  Byte hit rate: Percentage of data that is serviced using local cache. Why? Why?  ↑ cache hit rate means ↑ availability of data when a device has no base station coverage. How to manage cache? How to manage cache?  Offline techniques: Simple  Assumes advance knowledge of requests: Both “When?” and “what?”  Online techniques: LRU-K, GreedyDual Applies to techniq ues to manage a hierarc hy of memori es!

Simple [Ghandeharizadeh et. al. 2006] Offline technique: assumes the entire sequence of clip references is known in advance. Offline technique: assumes the entire sequence of clip references is known in advance. Compute frequency of access to clips. Compute frequency of access to clips. Freq-based algorithm: Freq-based algorithm:  Sort clips based on their frequency and assign the most frequently accessed clips to the device. Simple algorithm: Simple algorithm:  Sort clips based on their byte-freq value and assign those with highest value to a device. Byte-freq of clip j is defined as its frequency of access divided by its size.

Freq-based and Simple are NOT Optimal Freq-based and Simple are greedy heuristics. Freq-based and Simple are greedy heuristics.

Example Assume capacity of a device is 14 bytes. Assume capacity of a device is 14 bytes. Consider the shown repository: Consider the shown repository:  Simple assigns clip 2 to the device, resulting in a 10% hit ratio.  An optimal algorithm will assign clip 1 to the device, resulting in a 70% hit ratio. Clip-idSizeFreq Freq/siz e

Freq-based is not optimal Assume capacity of each node is 14 bytes. Assume capacity of each node is 14 bytes. Consider the shown repository: Consider the shown repository:  Frequency-based assigns clip 1 to a device, resulting in a 40% hit ratio.  Optimal assigns clips 2 and 3 to the device, resulting in a 60% hit ratio. Clip-idSizeFreqFreq/size

LRU [Denning 68] Online technique. Online technique. Victim is the least recently accessed clip. Victim is the least recently accessed clip. Pseudo-code for LRU (page p) If p is in the buffer then LAST(p) = current time; Else i) Min = current time + 1; ii) For all pages q in the buffer do a) If (LAST(q) < min) victim = q Min = LAST(q) iii) If victim is dirty then flush it to disk iv) Fetch p into the buffer frame held by victim v) LAST(p) = current time

LRU-K [O’Neils, Weikum 93] The victim page (page to be dropped) is the one whose backward K-distance is the maximum of all pages in buffer. The victim page (page to be dropped) is the one whose backward K-distance is the maximum of all pages in buffer. Definition of Backward K-distance bt(p,K): Given a reference string known up to time t (r1, r2, …,rt), the backward distance bt(p,K) is the distance backward to the K th most recent reference to page p. Definition of Backward K-distance bt(p,K): Given a reference string known up to time t (r1, r2, …,rt), the backward distance bt(p,K) is the distance backward to the K th most recent reference to page p.

Limitation of LRU-K Assumes equi-sized clips/pages. Assumes equi-sized clips/pages. Provides no support for variable sized clips. Provides no support for variable sized clips.

GreedyDual [Cao, Irani 97] Online technique Online technique Conceptually: Conceptually:  Assigns a priority Q to each clip.  When clip j is cached, Q is set to 1/Size(j).  When there is insufficient free space:  Victim is the clip with the lowest Q value (Q min )  For each cached clip j, reduce its Q j value by Q min.  If a clip k is referenced and observes a cache hit, restore its Q value to 1/Size(k)

GreedyDual Clever technique of [Cao, Irani 97] is simple and efficient. Clever technique of [Cao, Irani 97] is simple and efficient.

A Comparison: Cache Hit Rate Larger cache Variable sized objects.

Key Findings Simple and GreedyDual outperform LRU-K because they consider the size of clips when choosing a victim. Simple and GreedyDual outperform LRU-K because they consider the size of clips when choosing a victim. While memory is inexpensive, cache hit rate is a log function of memory size. While memory is inexpensive, cache hit rate is a log function of memory size.  One may have to increase memory size several folds in order to increase cache hit rate by a few percentage. GreedyDual is not a replacement for LRU-K with equi-sized pages. GreedyDual is not a replacement for LRU-K with equi-sized pages.  GreedyDual does not consider how recently clips were referenced.  E.g., buffer pool of 2 clips and the reference string: o 1, o 2, o 1, o 3, o 1, o 2, o 1, o 3,…

A Comparison: Byte Hit Rate Larger cache Variable sized objects.

A Comparison: Equi-Sized Objects Equi sized objects. Larger cache

Enhanced Online Techniques Is it possible to improve existing online caching techniques? Is it possible to improve existing online caching techniques?

Enhanced Online Techniques Is it possible to improve existing online caching techniques? Is it possible to improve existing online caching techniques?  Of course, the answer is YES. Otherwise, I would not be assigning this paper for you to read.

LRU-S2 Observation: LRU-K swaps out the clip with minimum value where I k is the interval of time from current time to the last K th reference. Observation: LRU-K swaps out the clip with minimum value where I k is the interval of time from current time to the last K th reference. LRU-SK swaps out clip with minimum value for: LRU-SK swaps out clip with minimum value for:

Improvement Larger cache

Online version of Simple. Key challenge: Compute the frequency of access to different clips online. Online version of Simple. Key challenge: Compute the frequency of access to different clips online.  Address the challenge using the history of references maintained for each clip (LRU-K). Conceptually: Conceptually:  Compute the arrival rate of requests for each clip i:  Frequency of access for clip j is its arrival rate divided by arrival rate of all other clips: Dynamic Simple (DYNSimple)

DYNSimple (Cont…) A larger value for K: A larger value for K:  Enhances estimates of frequency of access for a clip.  Makes it harder for DYNSimple to forget the past. DYNSimple won’t be able to react quickly to changing patterns of clip references. A value of 2 for K should be sufficient for most cases. A value of 2 for K should be sufficient for most cases.

Interval based GreedyDual (IGD) Extend IGD to consider interval of time since last reference. Extend IGD to consider interval of time since last reference. Conceptually: Conceptually:  Similar to LRU-K, maintain history of K references for each clip.  Let I K denote the interval of time for the past K references.  Let Ref(x) denote the number of references for clip x once it is cached, independent of its history.  Priority of clip x is set to:

Equi-sized clips

Zipfian distribution of access Visit for code Visit for codehttp://dblab.usc.edu

Varying Access Frequencies 10,000 requests are issued after each shift. 10,000 requests are issued after each shift. Clip Req Generator ClipID

Performance results

Adaptation

Adaptation (Cont…)

Contributions Identified limitation of today’s cache management techniques by comparing them with one another. Identified limitation of today’s cache management techniques by comparing them with one another. Three novel (online) techniques to enhance cache hit rate of a device: Three novel (online) techniques to enhance cache hit rate of a device: 1. DYNSimple 2. LRU-SK 3. Interval based GreedyDual (IGD) With K=2, DYNSimple provides a higher cache hite rate and adapts faster. With K=2, DYNSimple provides a higher cache hite rate and adapts faster.

Future Research Cooperative caching with Cooperative caching with 1. Stationary devices 2. Mobile devices

Stationary Devices A home network may consist of several wireless devices that communicate with one another. A home network may consist of several wireless devices that communicate with one another.  Example: An average household of 3 TVs uses one AppleTV to manage each TV. One or two PCs might also be deployed in the same household.

Stationary Devices

Should multiple devices cooperate and coordinate the state of their caches? Should multiple devices cooperate and coordinate the state of their caches? Why cooperate? Why cooperate?  Increase availability of data in the home  Minimize average startup latency, defined as the amount of time elapsed from when a user references a clip to the onset of display. Key design questions: Key design questions:  How does a device decide to join (leave) a cooperative group?  How is the autonomy of each device preserved when synchronizing its cache with other devices?  How is the placement of clips across the different caches controlled?

Cooperative Caching Consider multiple devices: Consider multiple devices:  Outside the coverage of the base station  Close enough to form an ad-hoc network Should these devices cooperate with one another by making their caches dependent on one another? Should these devices cooperate with one another by making their caches dependent on one another?

Final Thoughts Delivery of continuous media in wireless context is an exciting area of research with many challenging design decisions. Delivery of continuous media in wireless context is an exciting area of research with many challenging design decisions. Requirements of an application will dictate design decisions. Requirements of an application will dictate design decisions. Develop parameterized algorithms whose settings can support diverse applications. Develop parameterized algorithms whose settings can support diverse applications.