NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 1 Proxy Caching for Streaming Media

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 2 You Are Here Network Encoder Sender Middlebox Receiver Decoder

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 3 Types of Caches Browser cache For one user Proxy cache Shared cache between clients and server Gateway cache Content Delivery Networks (CDN) “Scale” server

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 4 Gateway Caches Deployed (or hired) by web site owners Makes sites more scalable and reliable Content is pushed out to caching nodes around the world Use DNS redirection to find closest cache Commercial CDNs: Akamai, Amazon CloudFront, …

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 5 Cache Proxies for Web A

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 6 Hierarchical Caching A B

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 7 Cooperative Caching AB

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 8 Distributed Caching AB

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 9 Streaming Media vs. Webpage

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 10 Video Access Pattern by S. Acharya and B. Smith in 1999 Study at Lulea University, Sweden 55% complete, 45% stop very early High temporal locality

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 11 Prefix Access Distribution

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 12 Video Popularity For modeling of video popularity (Note: this is not a measurement) Curve fitting to Zipf “Law” Probability of access to i-th most popular video can be approximated as:

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 13 Benefits of Caching

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 14 Reduce Access Latency :):(

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 15 Reduce Server Load

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 16 Reduce Start-up Latency

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 17 Hide Network Congestion

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 18 Other Issues What to cache? Who to fetch from? When cache is full, what to kick out? How to measure popularity? Can cache adapt to popularity?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 19 What to Cache?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 20 Segmentation Cache “all or none” is bad Divide media file into segments S and consider each segment individually

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 21 Effects of Segment Size S Large S : Low utilization Small S : Lots of gaps (fragmentation)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 22 Prefix Caching Policy 1 Chunk = k segments

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 23 Caching Policy Basic unit of caching: segment Cache prefix in chunk Replace suffix in chunk Never replace segments in currently accessed chunk

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 24 Where To Fetch From?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 25 Cooperative Caching AB

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 26 Fetch from Server B A Server Client 2Client 1

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 27 Fetch from Fellow Proxy B A Server Client 2Client 1

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 28 Issues How to advertise? How to choose “helper”?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 29 How to Advertise? Balance between network load freshness of information

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 30 Scalable Advertisement Expanding Ring Advertisement TTLPERIOD

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 31 How to Choose Helper? Consideration for Static Cache network distance (1,2,3,4) number of streams being served avoid frequent switches Build a cost function, integrating the metrics

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 32 Cost Function Cost for retrieving a segment from node X to node Y=

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 33 Algorithm Consider the next gap in local caches Find the next helper with minimum cost, which can fill in at least k segments in gaps

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 34 Distributed Caching Y. Chae et al. JSAC 2002

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 35 Cooperative vs. Distributed Cooperative caching caches independently Distributed caching caches as a team

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 36 Cold Start B A Server new clip!

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 37 Segment Map Local segment map Which segment should I cache? Global segment map Who is supposed to cache what?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 38 Cache Hit B A Server

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 39 Cache Miss B A Server

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 40 Distributed Caching B A Server

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 41 Problems Who should cache what? Which segment to kick out? How to adapt segment distribution?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 42 Who Should Cache What? RCache scheme Segment video into equal size segments A proxy will cache each segment with some probability

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 43 RCache N p proxies video of length L v divide into N s equal segments Each proxy caches each segment with a/N p probability

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 44 Analysis Probability that whole video is cached is N s: : num of segments N p: num of proxies a/N p: prob of caching 1 segment

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 45 RCache’s Segmentation Video is divided into segments of equal length Can we do better?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 46 Bimodal Distribution

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 47 Silo segment size probability of storage

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 48 Further Improvement segment size probability of storage

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 49 Problems Who should cache what? Which segment to kick out? How to redistribute data?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 50 Segment “Popularity” For each video i For each segment j F(i,j) = Prob(i is accessed)*Prob(j is accessed)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 51 Rainbow Algorithm less popular Segment Number

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 52 Rainbow Algorithm Problem: Many large video – too many segments computationally expensive to sort Solution: Just approximate by quantizing popularity

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 53 Rainbow Algorithm less popular

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 54 Problems Who should cache what? Which segment to kick out? How to redistribute data?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 55 Data Redistribution When popularity changes, need to redistribute. Redistribute “on-demand” (lazy)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 56 Cache Token Each segment have two bits (T,C) T: I am suppose to have the segment C: I have the segment

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 57 Data Redistribution (T=1,C=1) (T=0,C=0) (T=1,C=0) (T=0,C=1)

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 58 Example IJ ACBD old new T=0,C=1 T=1,C=0

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 59 Example IJ ACBD old new T=1,C=1 T=0,C=1 T=0,C=0T=1,C=0

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 60 Example IJ ACBD old new T=1,C=1 T=0,C=0T=1,C=0 T=0,C=0

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 61 Example IJ ACBD old new T=1,C=1 T=0,C=0T=1,C=1T=1,C=0T=0,C=0

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 62 Quality Adaptive Caching Reza Rajaie et al. INFOCOM 2000

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 63 Objective Quality adaptive streaming Use Scalable Video Coding (SVC) How to integrate with proxy caching?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 64 Scenario (10am) A Server Client 2

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 65 Scenario (1am) A Server Client 2

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 66 Problem: Caches interfere with congestion control algorithm Solution: Make cache aware of quality adaptation

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 67 Solutions Making cache “quality-aware” Prefetch Replacement Algorithm

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 68 Cache Miss A Server Client 2

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 69 Cache Hit A Server Client 2 repair + prefetch

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 70 Peeking Inside the Cache

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 71 Cache Hit: Repair

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 72 Cache Hit: Prefetch

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 73 Prefetch Algorithm prefetch window playback point

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 74 Proxy Request to Server Multiple requests (for different clients) are batched

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 75 Server Response Missing segments are sent in decreasing priority

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 76 Server Response Send as many segments as possible until next prefetch request

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 77 Trade-offs How far in the future should we prefetch?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 78 Solutions Making cache “quality-aware” Prefetch Replacement Algorithm

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 79 Goal: converge to efficient state If a stream is popular average quality is high variation in quality is low Goal of Replacement

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 80 The Algorithm

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 81 Thrashing and Locking

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 82 Choosing Victim whit (weighted hit) = T play /T total Calculate whit for each layer in a stream over a popularity window

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 83 Example WHITLock?NameLayer 5.91Nemo0 4.31Nemo1 4.00Matrix Matrix Gigli0

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 84 Static Cache Cache segments in proxy do not change over time Can we exploit further properties of streaming media?

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 85 Dynamic Caching

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 86 Motivating Scenario t = 0, R1 requests for stream M t = , R2 requests for stream M Ideally, R1 and R2 should share a multicast of M

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 87 needs to be “patched” share with R1 from cache

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 88 Using Dynamic Cache R2 request stream M Proxy allocate a ring buffer Cache the most recent  -seconds of M sent to R1 R2 get prefix of M from other places, and rest from proxy

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 89 Extending to N Receivers

NUS.SOC.CS Roger Zimmermann (based in part on slides by Ooi Wei Tsang) 90 Rules of Thumb Simple rules that should be considered in the design of caching architectures Cache must be large enough to hold “working set”, otherwise thrashing will happen Unified caches perform better than partitioned caches