1. Network CDNs Abhigyan Sharma Hello, I am Abhigyan from Umass Amherst.
I will talking about why geo-distributed Internet services must get their content placement right.

2. Geo-distributed, heterogeneous computing infrastructure
Cellular base stations
Set top boxes
Datacenters
Routers
Alice
Speed of light
Let’s start by imagining how computing infrastructure on the Internet is going to look like in the future.
Today, cloud-computing platforms with 10s of datacenter locations are pretty common.
Geo-distribution of datacenters is necessary because propagation delays on the Internet are fundamentally limited by the speed of light. A lower propagation delay between a user and its nearest datacenter improves user-perceived performance.
Given the importance of geo-distribution, we may see computing infrastructure with with 100s or even thousands of datacenters globally.
[pause]
Besides datacenters, there are other forms of computing resources that can emerge.
Examples are, set-top boxes, cellular base stations, as well as network routers, all of which could become platforms for general-purpose computation in future.
e.g., There are already some products in that enable general-purpose computation on network routers.
Thus, our future computing infrastructure will be both geo-distributed as well as heterogeneous.

3. Degrees of freedom enabled by geo-distribution✖ ✖ ✔
A key focus of this talk is geo-distributed infrastructure and services.
A geo-distributed infrastructure enables three degrees of freedom which services can leverage to enhance their performance.
So, let’s start with the simplest case.
Network routing
Content placement
Request redirection

4. Network routing/traffic engineering
Link capacity
Traffic
Engineering
Network
routing 4 2 9 6 7 5 8 1 3
Traffic from node i to node j
The third decision affecting geo-distributed systems is the network routing, or as it is often called traffic engineering.
Traffic engineering is a problem is solved by ISPs today to configure routing in their networks.
There are two inputs to the traffic engineering problem.
First, is the link capacity of all network links.
Next, is a traffic matrix. The i,j-th entry in this matrix is the traffic from node I to node j in the network topology.
Based on these inputs, traffic engineering computes a network routing, that is, the physical paths between all pairs of nodes in the network.
[pause]
In computing the routing, traffic engineering seeks to minimize cost functions …
Traffic matrix

5. Network cost function
Convex functions of link utilization, e.g., maximum link utilization (MLU) A B
1 Mbps
3 Mbps
1.5 Mbps
Shortest path
MLU = 2/3 = 0.67
Flow split
2 Mbps
0.5 Mbps
MLU = 0.5/1 = 0.5
The cost function for traffic engineering are commonly defined as the convex functions of link utilization.
For example, a common cost function is the maximum link utilization, MLU.
Traffic engineering seeks to minimize these cost function to achieve a lower link utilization.
A low utilization of all links in the network means that the network is free from congestion hotspots.
Further, a low link utilization means that the network has more spare capacity to tolerate sudden or long-term increase in demand.
We take two examples of routing schemes to show a careful routing can reduce link utilization.
One the left, we use a shortest path routing and send traffic via the lower

6. Leveraging geo-distribution: content placement
Placement vs. redirection
Alice
Placement vs. routing
Next, I will discuss why content placement is potentially a more powerful factor than routing or redirection to leverage the benefits of geo-distribution.
First, let us consider redirection.
Among the set of locations with the content, redirection chooses a location to send request.
If the content is placed only at a far-away location, irrespective of how redirection is done, a user will observe a high latency.
On the other hand, content placement can create more options to the redirection scheme to chooses a location that is nearby.
Now, let us consider routing.
Let us say we want to compute routing for this traffic matrix.
The flexibility of placing content gives us the power to change this traffic matrix itself. For example, if all content that a node in a network needs is placed at the same node, there is no traffic to be sent to any other node, then the traffic matrix only as entries whose value is 0.
There is no routing problem to be solved!
While this is an extreme example, and you cannot place content at all locations in the world. It shows the power of content placement over routing. 4 2 9 6 7 5 8 1 3
Place all content at all locations
Traffic matrix
No routing problem!

7. ISPs evolving to Network CDNs (NCDNs)
Content providers
Content delivery network (CDN)
NCDN
NCDN
NCDN
The traditional tripartite view of content delivery consists of three sets of entities:
Content providers who produce content and want to deliver it to end-users
Internet service providers (or ISPs) who provide connectivity to end-users
& finally, a content delivery network or a CDN which sits as an overlay over network providers delivering content via the network to end-users devices.
In recent times, this traditional tripartite view of content delivery is changing due to emergence of network CDNs or NCDNs.  An NCDN is a ISP that also operates a CDN on its network to deliver content to its customer end-users.
In other words, an NCDN gives you Internet connectivity and also accelerates delivery of your videos and downloads such as iTunes and Netflix using its infrastructure.
So, why are ISPs deploying NCDNs?
A big factor is the rise in Internet traffic driven by the huge ongoing growth in on-demand video traffic. From , Internet traffic is expected to grow by 3x of which 79% will be video traffic in 2018.
XXXXXXXXXXXXXXXXXXXslide edit
By deploying NCDNs, operators hope to use pervasive content caching to reduce the traffic on their network links. Another motivation is to monetize their CDN service, e.g., by offering on-demand video service to end-users.
Today, more than 30 ISPs across the globe have deployed NCDNs on their networks. This number is expected to increase because CDNs are selling their technology as a commodity to ISPs to build Network CDNs. These arrangements are called licensed CDN and managed CDN. If this trend continues, NCDNs could change the landscape of content delivery in the future Internet.
In this talk, I will pose the NCDN management problem that is the different from the problem of managing a traditional ISP or a traditional CDN.
And experimentally analyze multiple solutions to NCDN management problem.
Internet service providers (ISPs)
3X traffic growth (’13-18’), 79% video traffic (’18)*
*Cisco Visual Networking Index
CDNs commoditized: licensed & managed CDNs
30+ NCDNs

8. NCDN model Origin servers NCDN POP Backbone router Content servers
Backbone router at
exit nodes 8

9. NCDN management problem
Link capacity
NCDN
management
Storage capacity
Placement
Redirection
Routing
Content size vector , 4 2 9 6 7 5 8 1 3
Demand for content i at node j
***** say that this is content matrix different from traffic matrix *****
In slide we pose the NCDN management problem we address in our work.
Unlike TE which takes as input a traffic matrix, NCDN management takes as input a content matrix.
XXXXX also talk about static content here
The i.j-th entry in the content matrix shows the demand for a content I node J in the network.
In order to determine the content placement, there are two additional inputs.
First, the storage capacity at each node.
Second, is a vector that gives the size of each content in the content matrix.
NCDN seeks to optimize a cost function similar to traffic engineering, which could be TE cost function such as MLU or a cost function related to end-user performance, e.g., a latency cost function.
The output of the NCDN management problem are decisions regarding placement, redirection and routing.
A natural question is why not make these decisions independently, instead of make them jointly as this problem.
Content matrix

10. NCDN placement routing interaction
0.25 Mbps B C
0.5 Mbps
Traffic labeled with flow value
0.25 Mbps
8 Mbps
4 Mbps
1.25 Mbps
Link labeled with capacity
In this network, link capacities are as shown.
Lets say there is a content cache at node C where a single MPEG video file is stored. This video file has a demand of 1 Mbps at node A & 0.5 Mbps at node D, and there is no other traffic.
To minimize the maximum link utilization in this case, the traffic should be routed as follows, which results in an MLU of 0.5. To achieve this MLU, we need to use flow-split routing which splits traffic in the ratio 1:3 between the two paths CBA & CDA.
[pause]
Unlike the traditional TE model, where an ISP has no control over content placement, an NCDN has full flexibility to place content. Lets say an NCDN leverages this power to place content also at node B.
In this case, the demand at A & B can be served by sending traffic over links BA and CD. The MLU in this case is 0.125, which is 75% lower than the previous case. Also, we can use a shortest path routing with equal link weights to achieve this minimum MLU.
The point of this example is that content placement flexibility is powerful in shaping the network traffic. It can reduce network costs and enable simpler routing.
1.5 Mbps A D
0.75 Mbps
Demand = 1 Mbps
Demand = 0.5 Mbps
Maximum link utilization (MLU) = 0.75/1.5 = 0.5

11. NCDN placement routing interactionB C
0.5 Mbps
Traffic labeled with flow value
1 Mbps
8 Mbps
4 Mbps
0.5 Mbps
Link labeled with capacity
1.5 Mbps A D
Demand = 1 Mbps
Demand = 0.5 Mbps
Maximum link utilization (MLU) = 0.75/1.5 = 0.5
= 1/8 = 0.125

12. NCDN joint optimization
Joint optimization is NP-complete & inapproximable
Mixed integer program for
joint opt.
Joint optimization
Content
matrix
Routing
Placement, Redirection
Optimize:
Cost function
Constraints:
Link capacity
Storage capacity …
Origin servers
Optimize: cost function
Subject to:
Link capacity
Storage capacity …

13. Planned NCDN management
CM(t) : Content matrix in interval t
PRR(t) : Placement, redirection, routing in interval t
Planned: CM(t-1) PRR(t)
Oracle: CM(t) PRR(t)
Timeline
(t-1) t (t+1)
NCDN joint opt.
**** Transition: motivate complex, intractable for very large problem sizes, so therefore unplanned ****
Based on the joint optimization, we define a planned NCDN management scheme as follows.
Let us divide time into equal length intervals, and consider a time interval $t$.
We use CM(t) to denote the content matrix for the time interval t.
We use PRR(t) to denote the placement, redirection , routing, and for the time interval t.
The planned approach uses the content matrix CM(t-1) to compute PRR(t). That is it uses the CM in the immediately preceding interval to compute the placement redirection and routing for the current interval. This planned approach is a realistic way to perform joint optimization in NCDNs based on historically observed content demand.
Of course, instead of the CM(t-1), we could have considered older CM’s e.g., CM(t-2) as well. All of those fall under the planned approach.
Next, we define an ideal joint optimization scheme Oracle, which has future knowledge of content demand.
It uses the content matrix for the present interval CM(t) to compute PRR(t). Oracle is clearly unrealistic to implement, but it is useful to bechmark other schemes.
NCDN joint opt.

14. Unplanned NCDN management
Placement  LRU caching
Redirection  Closest hop count
Routing  Static shortest-path
Unplanned approach is simpler in that it does not require measurement of the content matrix in each interval which is necessary for planned approaches.
It uses simple heuristics to handle placement routing and redirection.
For content placement, .. Unlike planned approach it can continuously keep changing the content placement. …

15. Network cost Akamai traces: 7.2M users, 28.2M requests, 1455 TB data3x
*****Storage ratio constraint. Why? Motivate infinite storage there is no problem, but storage is not the only constraint *******
*****Representative of user-perceived latency cost*****
This slides compares the schemes based on their network cost.
The x-axis shows the storage ratio. It is the ratio of total storage at all POPs and the size of unique content observed on a typical day.
The y-axis shows the network cost function maximum link utilization (or MLU) normalized with respect to the highest MLU value.
As storage ratio increases, content can be replicated at more locations, which reduces network traffic and results in a lower MLU.
We find that a PLANNED JOINT OPTIMIZATION has a 3x higher MLU than the simple unplanned scheme. Why is this so? This is because PLANNED makes its decisions based on content demand over the previous day. All the Akamai traces we experiment with have a significant amount of daily churn in content popularity, and a significant fraction, up to 60%, of requests are for new content published that day. Therefore, a planned scheme needs to frequently fetch content from origin servers, which increases utilization of links along the path to the origin servers, which causes a high MLU.
We have also evaluated other variations of PLANNED such as combining PLANNED with UNPLANNED and running PLANNED scheme multiple times per day, but PLANNED has a higher network cost than UNPLANNED in those experiments as well.
On the other hand, the unplanned scheme performs well because its placement strategy, LRU caching, achieves high cache hit rates. We have seen cache hit rates in excess of 90% some traces. Due to its effective content placement, UNPLANNED achieves an MLU within 18% of ORACLE at a storage ratio of 4.
In addition to the network cost function MLU, we have also compared these schemes in terms of a latency cost function, our experiments show qualitatively similar findings in that experiment.
What these results show is that placement matters tremendously in NCDNs.
PLANNED JOINT-OPTIMIZATION is poor at content placement, so it performs worse.
UNPLANNED scheme works well because its placement scheme LRU caching works well.
Next question is …
18%

16. User-perceived latency cost
Latency Cost = E2E propagation delay + Link utilization dependent delay
Content placement matters tremendously in NCDNs
Transition:
Lets go back to an earlier question I had raised.
In this experiment, we evaluate the user perceived latency for different schemes.
An equally important metric for an NCDN is the latency experienced by end-users, which we evaluate in this experiment.
To calculate latency, we have defined a latency cost function which captures the end-to-end delay including propagation delay and delay caused by retransmissions after a packet loss.
The value of the latency cost function is shown on the y-axis. X-axis is the storage ratio.
The relative performance of schemes reflects a similar trend with the latency cost function as with the network cost function.
Joint-Opt strategy has a high latency cost for two reasons, first its higher MLU causes more retransmissions, and second it frequently needs to fetch content from origin servers which also increases latency.
InvCap-LRU performs better than a Joint-Opt strategy. InvCap-LRU improves its performance relative to the Joint-Opt-Future scheme on increasing the storage ratio. At a storage ratio of 4, the latency of InvCap-LRU is within 28% of Optimal while than Joint-Opt strategy has a 70% higher latency cost than optimal.
This experiment shows than an unplanned scheme has a close to optimal latency cost at higher storage ratios.
What these experiments placement matters in NCDNs.
JOINT-OPTIMIZATION poor at content placement: performs worse.
UNPLANNED placement works well, and performs well.
Next question is does routing matter?
28%

17. Traffic engg. vs. static shortest-path routing
LRU Caching + Static short-path routing
vs.
LRU Caching + Traffic engineering
Traditional traffic engg. gives small cost reduction in NCDNs
10% or less
… how much does routing matter?
XXXXXXXXXXXX
We evaluate the relative importance of optimizing content placement vs. that of optimizing routing. The graph shows how much does network cost reduce if we use a planned TE instead of Unplanned TE along with an unplanned LRU placement.
The y-axis on this graph shows the maximum reduction that a planned TE achieves over the unplanned static routing scheme.
The x-axis shows the results for various traces.
We find that the maximum reduction is less than 10%, suggesting that a planned routing brings only a small improvement over UNPLANNED.
The interaction between content delivery and network routing is of …

18. Message to NCDNs Keep it simple
Realistic joint optimization performs worse than simple unplanned
Little room for improvement over simple unplanned
Content placement matters more than routing in Network CDNs
Planned strategies perform poorly, even hybrid strategy and other demand-aware schemes
Simple demand-aware aware schemes perform surprisingly well, and on provisioning more storage Content placement is useful

19. Traffic engg. schemes comparison for random placement
All traffic engg. schemes achieve near-optimal capacity.
Static shortest-path routing at most 30% sub-optimal.
Before moving on, I will briefly present a result from another piece of my work related to NCDNs.
In this work, we consider an ISP network in which all content is placed at a few randomly chosen locations.
End-users download each content in parallel from all locations using TCP connections.
Our goal is to evaluate how does a random content placement at multiple locations affect the performance of common traffic engineering schemes.
Our finding is that having content at just 2 or 4 locations blurs the difference between traffic engg. Schemes, and all traffic engg. schemes are near-optimal on a capacity metric.
And the gap between optimal and static shortest-path routing is at most 30%.
So, it is not surprising that, in an NCDN, where placement is better enginered than a random placement, the gap between traffic engg and static shortest-path routing further reduces to 10%.
INFOCOM 2011 19