Smart Content Delivery in Large Networks: En-Route Caching Hong Shen School of Computer Science University of Adelaide, Australia Dept. of Computer Sci. & Tech. University of Sci. & Tech. of China

Outline of the Talk Problem formulation Unconstrained solution Constrained solutions Solution for m servers

Content Distribution Network Sits between content providers and content consumers. Contains hundreds of servers throughout Internet. Replicates and maintains customers’ content in CDN servers.

CDN Example: Google platform Maintains over 450,000 CDN servers, arranged in racks located in clusters in cities around the world Allows users to access its content most rapidly by sending them lightly loaded and geographically proximate servers.

Bottleneck of CDNs Multiple transmission flows for the same object. Solution: caching the object in selected nodes. WHEN and HOW? Challenges

En-Route Object Caching Object caching: Store most commonly accessed objects close to clients En-route object caching: Objects are cached at selective nodes on the access path from client to server Server Hold no copy Hold a copy request object

En-Route Object Caching (cont.) Why en-route? Important observation: Users normally have regular access patterns; Storing object at en-route nodes during delivery does not consume extra bandwidth.

Caching Performance The performance of en-route object caching depends mainly on two factors: The locations of the caches (Cache Location) The management of the cache contents (Content Replacement) Coordinated Caching: Consider both factors when making cache decision.

Our Work Web object en-route caching in tree networks ACM Transactions on Internet Technology, Vol. 5, No. 3, 2005, p. 480-507.   Multimedia object en-route caching in tree networks ACM Transactions on Multimedia Computing, Communications and Applications, Vol. 1, No. 3, 2005, p. 289-314. Multimedia object placement for transparent data replication in linear array IEEE Transactions on Parallel & Distributed Systems, Vol. 18 , No. 2, 2007, p. 212-224. Multiserver en-route web caching IEEE Transactions on Computers (under review), 2007.

Definitions and Notations G=(V,E) is a graph, where V is the set of nodes and E is the set of links. Cost saving s(v): the cost saving of storing a new object in node (cache) v. Cost loss l(v): the cost loss of removing other objects from node v in order to accommodate the new object. Cost gain g(v): g(v)=s(v) –l(v).

Problem Formulation G(P)= Find a node set P to store the object s.t. the total cost gain is maximized: G(P)=

Problem Formulation for Tree Networks G Aw w D(w) Server v’ Hold no copy Hold a copy v f(v) f’(v)

Constraints The different cases of C include: C is null (unconstrained). The cost gain for each node is greater than zero, i.e., g(v)>0 for all v in P. The number of copies is exactly k, i.e., |Aw|=k. The number of copies is no more than k, i.e. |Aw| k.

Solution for Unconstrained Case Main idea: Decompose the tree level by level recursively to a set of lines or singletons (nodes) whose solutions are known. Solution (Aw) to tree Tw is obtained by combining (union of) the solutions (Aw,x) to Tw’s subtrees.

Tree Decomposition (1) C(w): set of all children of node w.

 Decomposition of A * w Aw,w2 Aw,w1 * w Aw w * w1 w2 w2 w1 Theorem 1 shows that the optimal solution for the whole tree can be divided into the optimal solutions for some subtrees according to the children of the root. For this example, since the root has two children, the optimal solution for the whole tree can be obtained by the optimal solutions for the two subtrees on the left.

Tree Decomposition (2)

 Decomposition of A  G( ) G( ) A A A T * w, x Theo. 2 1 2 Theo.1  ,, w, A x * ,,1 w, A ,, T , G( ) ,,2  G( ) {x}  1 x x x 1 2 2. otherwise 2 1.

Algorithm 1

Algorithm 1: Continued

Time Complexity The algorithm runs in time: tw= O( vC(w) ( C(v)+tv) ) = O(vVD(v)) = O(n2), where n is the total number of nodes in the network.

Solution for Constrained Case I Non-negative cost gain per node (1)

Transformation The optimal solution for Problem (1) is equivalent to (2)

Algorithm 2

Algorithm 2 (Continued) Time Complexity: O(n2)

Solution for Constrained Case II Placing exactly k copies (3)

Algorithm 3 Time Complexity: O(n2log(fn)), where f=max{f(v)}.

Solution for Constrained Case III Placing at most k copies (4)

Algorithm 4 Time Complexity: O(kn2log(fn)), where f=max{f(v)}.

Extension to ASes System Model:

Solution Dividing the whole system into two parts and one part is a tree. Continuing to divide the other part in the same way until there is only one tree left. Applying the methods for tree network.

More General Setting: m-Sever En-route Caching A set of servers S={sj, 1≤ j ≤m} located at leaves of a tree. Cost saving for node w, s(w, dj), under the condition that the distances from w to the nearest high level node towards server sj that holds a copy is dj. Find a node set P to store the object, s.t. the total gain is maximized (vP serves nodes g(v,S)) 32

The Challenge We can’t get optimal solution to multi-server problem by simply combining solutions to 1-server problem. + A Simple 2-Server Problem Solve 1-server problem Optimal Solution ≠ Hold a copy No copy 33

A More General Definition Condition Dw, Dw=[d1,…dj,…dm], dj is the distance from node w to the nearest node towards sj, for example u, that hold a copy of object O. G(w, Dw), is the objective value of (6) in Tw under condition Dw, A(w, Dw) is the solution corresponding to G(w, Dw). 34

Lemma 1 For tree Tr containing m servers at leave nodes, the distances from wi to the nearest node towards sj that holds a copy are denoted by e(wi,dj) and k(wi,dj) for the cases that node wi holds a copy and no copy respectively, then we have r s1 s2 s3 wi ∈ path[r, sj] means server sj is in the sub-tree twi, because servers are located at leaves. An example of multi-server network 35

Theorem 3 For tree Tr containing m servers at leave nodes, the optimal solution of (6) is A(r, Dr) and corresponding objective value is G(r, Dr), where Dr is the vector of distances from root node to servers and 36

Theorem 3 (cont.)

Time complexity of the algorithm is O(nhm). Main idea: Problem is split top-down and solution A(r, Dr) is generated bottom-up according to Theorem 3, with corresponding objective value G(r, Dr). Time complexity: Algorithm computes all G(w, Dw), where w∈ V, Dw = [d1,…dj,…dm], 0 ≤ dj ≤ hw, hw is the distance from w to sj, hw ≤ 2h. Time complexity of the algorithm is O(nhm). 38

Conclusion New tree decomposition techniques for en-route web caching. Polynomial-time algorithms for the first time for 1-server en-route web-caching in tree networks. p-server en-route web caching in tree networks: O(nhm ) time.

Questions?

Calculating cost loss l(v) Cost loss l(v): The additional cost caused by removing some objects from v to make room for the new object: Holding no copy Server Missing penalty m(v): The additional cost of accessing the object if it is not cached at v. E.g. m(3)=c(3,0), m(7)=c(7,4). Holding a copy 2 1 c(3,0) 5 f’(3)=0 f’(4)=f(6) f’(5)=f(8) +f(9) 3 4 c(9,5) 6 7 8 9