Resource Placement and Assignment in Distributed Network Topologies Accepted to: INFOCOM 2013 Yuval Rochman, Hanoch Levy, Eli Brosh.

Slides:



Advertisements
Similar presentations
Hadi Goudarzi and Massoud Pedram
Advertisements

Class-constrained Packing Problems with Application to Storage Management in Multimedia Systems Tami Tamir Department of Computer Science The Technion.
Techniques for Dealing with Hard Problems Backtrack: –Systematically enumerates all potential solutions by continually trying to extend a partial solution.
1 Efficient and Robust Streaming Provisioning in VPNs Z. Morley Mao David Johnson Oliver Spatscheck Kobus van der Merwe Jia Wang.
Modeling Rich Vehicle Routing Problems TIEJ601 Postgraduate Seminar Tuukka Puranen October 19 th 2009.
Five Problems CSE 421 Richard Anderson Winter 2009, Lecture 3.
1 s-t Graph Cuts for Binary Energy Minimization  Now that we have an energy function, the big question is how do we minimize it? n Exhaustive search is.
Optic fiber Electronic switch the fiber serves as a transmission medium Optical networks - 1 st generation 1. Optical networks – basic notions.
The Cache Location Problem IEEE/ACM Transactions on Networking, Vol. 8, No. 5, October 2000 P. Krishnan, Danny Raz, Member, IEEE, and Yuval Shavitt, Member,
Item Pricing for Revenue Maximization in Combinatorial Auctions Maria-Florina Balcan, Carnegie Mellon University Joint with Avrim Blum and Yishay Mansour.
Beneficial Caching in Mobile Ad Hoc Networks Bin Tang, Samir Das, Himanshu Gupta Computer Science Department Stony Brook University.
Online Algorithms for Network Design Adam Meyerson UCLA.
Greedy Algorithms Reading Material: Chapter 8 (Except Section 8.5)
Global Synchronization in Sensornets Jeremy Elson, Richard Karp, Christos Papadimitriou, Scott Shenker.
The max flow problem
Maximum Flows Lecture 4: Jan 19. Network transmission Given a directed graph G A source node s A sink node t Goal: To send as much information from s.
Distributed Servers Architecture for Networked Video Services S. H. Gary Chan, Member IEEE, and Fouad Tobagi, Fellow IEEE.
Minimum Cost Flow Lecture 5: Jan 25. Problems Recap Bipartite matchings General matchings Maximum flows Stable matchings Shortest paths Minimum spanning.
A General Approach to Online Network Optimization Problems Seffi Naor Computer Science Dept. Technion Haifa, Israel Joint work: Noga Alon, Yossi Azar,
A Customizable k-Anonymity Model for Protecting Location Privacy Written by: B. Gedik, L.Liu Presented by: Tal Shoseyov.
1 The Min Cost Flow Problem. 2 The Min Cost Flow problem We want to talk about multi-source, multi-sink flows than just “flows from s to t”. We want to.
CUHK Analysis of Movie Replication and Benefits of Coding in P2P VoD Yipeng Zhou Aug 29, 2012.
Tradeoffs in CDN Designs for Throughput Oriented Traffic Minlan Yu University of Southern California 1 Joint work with Wenjie Jiang, Haoyuan Li, and Ion.
1 Distributed Computing Optical networks: switching cost and traffic grooming Shmuel Zaks ©
A User Experience-based Cloud Service Redeployment Mechanism KANG Yu.
The Theory of NP-Completeness 1. Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking.
Escape Routing For Dense Pin Clusters In Integrated Circuits Mustafa Ozdal, Design Automation Conference, 2007 Mustafa Ozdal, IEEE Trans. on CAD, 2009.
Bargaining Towards Maximized Resource Utilization in Video Streaming Datacenters Yuan Feng 1, Baochun Li 1, and Bo Li 2 1 Department of Electrical and.
Steady and Fair Rate Allocation for Rechargeable Sensors in Perpetual Sensor Networks Zizhan Zheng Authors: Kai-Wei Fan, Zizhan Zheng and Prasun Sinha.
DaVinci: Dynamically Adaptive Virtual Networks for a Customized Internet Jennifer Rexford Princeton University With Jiayue He, Rui Zhang-Shen, Ying Li,
Internet Traffic Engineering by Optimizing OSPF Weights Bernard Fortz (Universit é Libre de Bruxelles) Mikkel Thorup (AT&T Labs-Research) Presented by.
 1  Outline  stages and topics in simulation  generation of random variates.
1 On the Placement of Web Server Replicas Lili Qiu, Microsoft Research Venkata N. Padmanabhan, Microsoft Research Geoffrey M. Voelker, UCSD IEEE INFOCOM’2001,
June 21, 2007 Minimum Interference Channel Assignment in Multi-Radio Wireless Mesh Networks Anand Prabhu Subramanian, Himanshu Gupta.
1 CS612 Algorithms for Electronic Design Automation CS 612 – Lecture 8 Lecture 8 Network Flow Based Modeling Mustafa Ozdal Computer Engineering Department,
EE 685 presentation Utility-Optimal Random-Access Control By Jang-Won Lee, Mung Chiang and A. Robert Calderbank.
Zibin Zheng DR 2 : Dynamic Request Routing for Tolerating Latency Variability in Cloud Applications CLOUD 2013 Jieming Zhu, Zibin.
1 Distributed Computing Optical networks: switching cost and traffic grooming Shmuel Zaks ©
Mohamed Hefeeda 1 School of Computing Science Simon Fraser University, Canada Efficient k-Coverage Algorithms for Wireless Sensor Networks Mohamed Hefeeda.
1/24 Introduction to Graphs. 2/24 Graph Definition Graph : consists of vertices and edges. Each edge must start and end at a vertex. Graph G = (V, E)
Max Flow – Min Cut Problem. Directed Graph Applications Shortest Path Problem (Shortest path from one point to another) Max Flow problems (Maximum material.
1 Statistical Modeling and Analysis of P2P Replication to Support Vod Service zyp Infocom, 2011, Shanghai.
CS223 Advanced Data Structures and Algorithms 1 Maximum Flow Neil Tang 3/30/2010.
ASSIGNMENT, DISTRIBUTION AND QOS PROVISIONING IN COMMUNICATION NETWORKS.
CSCI-256 Data Structures & Algorithm Analysis Lecture Note: Some slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. 25.
1 CS612 Algorithms for Electronic Design Automation CS 612 – Lecture 8 Lecture 8 Network Flow Based Modeling Mustafa Ozdal Computer Engineering Department,
Chapter 7 May 3 Ford-Fulkerson algorithm Step-by-step walk through of an example Worst-case number of augmentations Edmunds-Karp modification Time complexity.
Distributed, Self-stabilizing Placement of Replicated Resources in Emerging Networks Bong-Jun Ko, Dan Rubenstein Presented by Jason Waddle.
On the Placement of Web Server Replicas Yu Cai. Paper On the Placement of Web Server Replicas Lili Qiu, Venkata N. Padmanabhan, Geoffrey M. Voelker Infocom.
Optimizing server placement in distributed systems in the presence of competition Jan-Jan Wu( 吳真貞 ), Shu-Fan Shih ( 施書帆 ), Pangfeng Liu ( 劉邦鋒 ), Yi-Min.
Prof. Swarat Chaudhuri COMP 382: Reasoning about Algorithms Fall 2015.
The Theory of NP-Completeness 1. Nondeterministic algorithms A nondeterminstic algorithm consists of phase 1: guessing phase 2: checking If the checking.
Flows and Networks (158052) Richard Boucherie Stochastische Operations Research -- TW wwwhome.math.utwente.nl/~boucherierj/onderwijs/158052/ html.
Learning Deep Generative Models by Ruslan Salakhutdinov
Procrastination … with Variable Present Bias
What is VN Embedding? Given: Task :
A Study of Group-Tree Matching in Large Scale Group Communications
Efficient Policy-Based Routing without Virtual Circuits
Maximum Matching in the Online Batch-Arrival Model
Chapter 5. Optimal Matchings
1.3 Modeling with exponentially many constr.
Edmonds-Karp Algorithm
Introduction Basic formulations Applications
Course Contents: T1 Greedy Algorithm Divide & Conquer
Replications in Multi-Region Peer-to-peer Systems
Problem Solving 4.
1.3 Modeling with exponentially many constr.
Flow Networks and Bipartite Matching
Chapter 1. Formulations.
Replications in Multi-Region Peer-to-peer Systems
Presentation transcript:

Resource Placement and Assignment in Distributed Network Topologies Accepted to: INFOCOM 2013 Yuval Rochman, Hanoch Levy, Eli Brosh

Motivation: Video-on-Demand service  Video-on-Demand (VoD) internet service  Large collection of movies  Highly-variable  Geo-distributed demand  Use Content Distribution Network 2 Rochman, Levy, Brosh April 2013

Motivation: Content Distribution Network  Multi-region server structure (e.g., terminal based service, cloud)  Service costs: intra-region < inter-region < central 3 Rochman, Levy, Brosh April 2013 Intra-region Low cost Inter-region Medium cost Central High cost Region 2 Central video server Region 1 User terminals - demand Request type Disk

System and Objective  Players: users + content servers (local, central)  Objective: Reduce service costs  Replicating content at regions 4 Central video server Region 1 Region 2 User terminals - demand Low cost Medium cost High cost  Problem: Which movies to place where? Rochman, Levy, Brosh April 2013

 Tewari & Kleinrock [2006]  Proposed the Proportional Mean Replication.  Zhou, Fu & Chiu [ 2011]  Proposed the RLB (Random with Load Balancing) Replication. Related Work 5 Rochman, Levy, Brosh April 2013

The Multi-Region Placement Problem 6 Available resource Local storage Input:  Region j storage size: S j  Stochastic demand distribution N i j, random variable.  Service costs Rochman, Levy, Brosh April 2013 S 1 =4 S 2 =2 ? ? ? ? ? ? ? ? Pr(N 1 1 <=x) Pr(N 2 1 <=x) Pr(N 1 2 <=x)Pr(N 2 2 <=x) ? ? ? ? Stochastic demand E.g., high-variability, correlated Local < Remote < Server

Pr(N 1 1 <=x) Pr(N 2 1 <=x) Pr(N 1 2 <=x)Pr(N 2 2 <=x) S 1 =4 The Multi-Region Placement Problem 7 Local < Remote < Server Input: Storage S j, demand N i j, service costs Allocation: Place resources at regions Cost of allocation: expected cost of optimal assignment (over all demand realizations) Goal: find allocation with minimal cost Rochman, Levy, Brosh April 2013 Actual demand S 2 =2 ? ? ? ? Stochastic demand Available resource ? ? ? ? ? ? ? ? Local storage

Challenge and principles Challenge: Combinatorial problem based on multi- dimensional stochastic variables Keys of solution: Semi-Separability, Concavity, Reduction to Min-cost Flow problem. 8 Rochman, Levy, Brosh April 2013 Local storage S 1 =4 S 2 =2 ? ? ? ? ? ? ? ? Pr(N 1 1 <=x) Pr(N 2 1 <=x) Pr(N 1 2 <=x)Pr(N 2 2 <=x) ? ? ? ? Stochastic demand Available resource Exponential number of allocations Large database!

Single Region: Matching Demand realization to resources Observed Demand Resources A profit formula! 9 Rochman, Levy, Brosh April 2013

Single region: Revenue Formulation  Lemma: optimal matching maximizes revenue of a realization 10 Random Demand Type-i replicas  Hence: we have to maximize For any placement and demand Rochman, Levy, Brosh April 2013

Multi-Region: Matching Match local first, then remote, then server. 11 Rochman, Levy, Brosh April 2013 Available resource

Multi-Region: Revenue formulation  Thm: maximize revenue to find opt placement {L i j } : Local revenue Global revenue Type-i resources at region j 12 Rochman, Levy, Brosh April 2013 Local=3 Global=4

Separability and semi-Separability  Definition: function is separable iff  Sum of separated marginal components  Definition: function is semi-separable iff  “Almost” separated components 13 Rochman, Levy, Brosh April 2013 Where

Key 1: Revenue is Semi-separable  Revenue function  Revenue function is semi-separable.  Sum of local replicas = # global replicas. 14 Rochman, Levy, Brosh April 2013 Local replicas Global replicas

Key 2: Concavity  Partial expectation  Partial Expectations are concave!  Cumulative(cdf) is monotonic  Thus, Partial expectation is concave 15 Rochman, Levy, Brosh April 2013 Tail formula:

Placement Optimization Problem 16  Find {L i j } allocation of type-i movie at region j ({L i j } ) maximizing:  Under: capacity bound in each region Concave in placement vars {L i j } Rochman, Levy, Brosh April 2013

The Multi-Region Problems 17  Symmetric bounded– QEST 2012, low complexity  Greedy algorithm, max-percentile based  Asymmetric bounded– INFOCOM 2013, higher complexity  Reduction to min-cost flow problem Rochman, Levy, Brosh April 2013

Key 3: Min cost flow 18 Rochman, Levy, Brosh st 11/13 12/12 15/20 1/41/4 4/94/9 7/77/7 4/44/4 8/13 11/ Flow/Capacity 0 Weight April 2013 Flow value Flow weight (cost)

The Min-Cost Flow Problem 19Rochman, Levy, Brosh  Input:  A positive capacity function C on the edges, C: E  R +  A positive weight function W on the edges, W: E  R +  Required Flow value r  Output: : an s-t flow f, with flow value= r, which minimizes weight Σf(e) W(e). st 11/13 15/20 1/41/4 4/94/9 7/77/7 4/44/4 8/13 11/ /12 April 2013

Main theorem 20Rochman, Levy, Brosh Theorem :  Assume: - concave & semi-separable  Then, there is effective solution for  Solution uses min cost flow algorithm  On 7-layer graph!  Correctness at the paper. April 2013

7-layer graph: Local part S Region 21 Region, Movie type Region, Movie, # replicas Capacity, Weight Rochman, Levy, Brosh April 2013 Capacity of region Local weight

7-layer graph: Global part t Region, Movie type, # items Movie type, # items Movie type 22 April 2013 Rochman, Levy, Brosh Global weight

Min-Cost Flows  Standard solution to min-cost flow using Successive Shortest Path (SSP).  Complexity of SSP (standard solution) is  s= total storage in the system  k= # regions  m= # movie types  High complexity! 23Rochman, Levy, Brosh April 2013

Other proposed algorithms  Bipartite algorithm (INFOCOM 2013) in complexity of (instead of )  Idea: use only Region and movie type nodes  Clique algorithm -complexity of  Online algorithm. 24Rochman, Levy, Brosh s= total storage in the system k= # regions m= # movie types April 2013

Conclusions  Algorithms for resource placement and assignment  Geared for distributed network settings  Arbitrary demand pattern (e.g., highly-variable, correlated)  Joint placement-assignment problem  Multi-dimensional stochastic demand  New solution techniques 25 Rochman, Levy, Brosh April 2013

Questions? 26Rochman, Levy, Brosh April 2013

 An alternative allocation: Proportional mean  Allocate movies proportion to mean of distribution How good are the results? 27 Rochman, Levy, Brosh April 2013

 Two resource-types.  Single region, capacity n  Proportional Mean:  Expected profit= 2*n/(k+1)  Optimal allocation: n replicas to red.  Expected profit=n. Proportional Mean Not optimal n Pr(N=x) 1-1/k 1/k nk 2 x= Rochman, Levy, Brosh April 2013 demand

Reduction to single region S t.... Capacity, Weight Movie typeMovie type, # replicas 29Rochman, Levy, Brosh April 2013 Flow value= s

 Convert max to min Correctness 30Rochman, Levy, Brosh Original New If solution is April 2013

Correctness S t.... Capacity, Weight Movie type, # replicas 31Rochman, Levy, Brosh April 2013 < < Concavity! <

Reduction to multi region 32Rochman, Levy, Brosh  Convert max to min: Global Local April 2013 Semi-Separability!