1. Approximation Algorithms: Éva Tardos Cornell University problems, techniques, and their use in game theory

2. FOCS 20022 What is approximation? Find solution for an optimization problem guaranteed to have value close to the best possible. How close? additive error: (rare) –E.g., 3-coloring planar graphs is NP-complete, but 4-coloring always possible multiplicative error: –  -approximation: finds solution for an optimization problem within an  factor to the best possible.

3. FOCS 20023 Why approximate? NP-hard to find the true optimum Just too slow to do it exactly Decisions made on-line Decisions made by selfish players

4. FOCS 20024 Outline of talk Techniques: Greedy Local search LP techniques: rounding Primal-dual Problems: Disjoint paths Multi-way cut and labeling network design, facility location Relation to Games – local search  price of anarchy – primal dual  cost sharing

5. FOCS 20025 Max disjoint paths problem Given graph G, n nodes, m edges, and source-sink pairs. Connect as many as possible via edge-disjoint path. t s t s s t t s

6. FOCS 20026 Greedy Algorithm Greedily connect s-t pairs via disjoint paths, if there is a free path using at most m ½ edges: m ½  4 s t s s t t s t If there is no short path at all, take a single long one.

7. FOCS 20027 Greedy Algorithm Theorem: m ½ –approximation. Kleinberg’96 Proof: One path used can block m ½ better paths m ½  4 s t s s t t s t Essentially best possible: m ½-  lower bound unless P=NP by [Guruswami, Khanna, Rajaraman, Shepherd, Yannakakis’99]

8. FOCS 20028 Disjoint paths: open problem Connect as many as pairs possible via paths where 2 paths may share any edge t s t s s t t s Same practical motivation Best greedy algorithm: n ½ - (and also m 1/3 -) approximation: Awerbuch, Azar, Plotkin’93. No lower bound …

9. FOCS 20029 Outline of talk Techniques: Greedy Local search LP techniques: rounding Primal-dual Problems: Disjoint paths Multi-way cut and labeling network design, facility location Relation to Games – local search  Price of anarchy – primal dual  Cost sharing

10. FOCS 200210 Multi-way Cut Problem Given: –a graph G = (V,E) ; –k terminals { s 1, …, s k } –cost w e for each edge e Goal: Find a partition that separates terminals, and minimizes the cost  {e separated } w e Separated edges s1s1 s2s2 s3s3 s4s4

11. FOCS 200211 Greedy Algorithm For each terminal in turn –Find min cut separating s i from other terminals The first cut The next cut s2s2 s1s1 s4s4 s3s3 s2s2 s1s1 s4s4 s3s3

12. FOCS 200212 Theorem: Greedy is a 2-approximation Proof: Each cut costs at most the optimum’s cut [Dahlhaus, Johnson, Papadimitriou, Seymour, and Yannakakis’94] Cuts found by algorithm: Optimum partition Selected cuts, cheaper than optimum’s cut, but each edge in optimum is counted twice. s4s4 s3s3 s2s2 s1s1

13. FOCS 200213 Multi-way cuts extension Given: –graph G = (V,E), w e  0 for e  E –Labels L= {1,…,k} –L v  L for each node v Objective: Find a labeling of nodes such that each node v assigned to a label in L v and it minimizes cost  {e separated} w e Separated edges part 1 part 2 part 3 part 4

14. FOCS 200214 Example Does greedy work? For each terminal in turn –Find min cut separating s i from other terminals Blue or green Red or green Red or blue cheap medium expensive s3s3 s1s1 s2s2

15. FOCS 200215 Greedy doesn’t work Greedy For each terminal in turn –Find min cut separating s i from other terminals The first two cuts: Remaining part not valid! Blue or green Red or green Red or blue s2s2 s1s1 s3s3

16. FOCS 200216 Local search [Boykov Veksler Zabih CVPR’98] 2-approximation 1.Start with any valid labeling. 2. Repeat (until we are tired): a.Choose a color c. b. Find the optimal move where a subset of the vertices can be recolored, but only with the color c. (We will call this a c-move.)

17. FOCS 200217 A possible -move Thm [Boykov, Vekler, Zabih] The best -move can be found via an (s,t) min-cut

18. FOCS 200218 Idea of the flow network for finding a -move s = all other terminals: retain current color s c = change color to c = G

19. FOCS 200219 Theorem: local optimum is a 2-approximation Partition found by algorithm: Cuts used by optimum The parts in optimum each give a possible local move:

20. FOCS 200220 Theorem: local optimum is a 2-approximation Partition found by algorithm: Possible move using the optimum Changing partition does not help  current cut cheaper Sum over all colors: Each edge in optimum counted twice

21. FOCS 200221 Metric labeling  classification open problem Given: –graph G = (V,E); w e  0 for e  E –k labels L –subsets of allowed labels L v –a metric d(.,.) on the labels. Objective: Find labeling f(v)  L v for each node v to minimize  e=(v,w) w e d(f(v),f(w)) Best approximation known: O(ln k ln ln k) Kleinberg-T’99

22. FOCS 200222 Outline of talk Techniques: Greedy Local search LP techniques: rounding Primal-dual Problems: Disjoint paths Multi-way cut and labeling network design, facility location Relation to Games – local search  Price of anarchy – primal dual  Cost sharing

23. FOCS 200223 Using Linear Programs for multi-way cuts Using a linear program = fractional cut  probabilistic assignment of nodes to parts ? Idea: Find “optimal” fractional labeling via linear programming Label ? as : ½ + ½

24. FOCS 200224 Fractional Labeling Variables: 0  x va  1 p=node, a =label in L v –x va  fraction of label a used on node v Constraints:  x va = 1 aLvaLv for all nodes v  V – each node is assigned to a label cost as a linear function of x:  w e ½  |x ua - x va | e=(u,v) aLaL

25. FOCS 200225 From Fractional x to multi-way cut The Algorithm (Calinescu, Karloff, Rabani, ’98, Kleinberg-T,’99) While there are unassigned nodes select a label a at random x va 1 uvUnassigned nodes

26. FOCS 200226 The Algorithm (Cont.) While there are unassigned nodes –select a label a at random x va 1 uv Unassigned nodes  select 0    1 at random assign all unassigned nodes v to selected label a if x va  

27. FOCS 200227 Why Is This Choice Good? select 0    1 at random assign all unassigned nodes v to selected label a if x va   Note: Probability of assigning node v to label a is  x va Probability of separating nodes u and v in this iteration is  |x ua – x va | x pa 1 pq Unassigned nodes 

28. FOCS 200228 From Fractional x to Multi-way cut (Cont.) Theorem: Given a fractional x, we find multi-way cut with expected separation cost  2 (LP cost of x ) Corollary: if x is LP optimum.  2-approximation Calinescu, Karloff, Rabani, ’98 1.5 approximation for multi-way cut (does not work for labeling) Karger, Klein, Stein, Thorup, Young’99 improved bound  1.3438..

29. FOCS 200229 Outline of talk Techniques: Greedy Local search LP techniques: rounding Primal-dual Problems: Disjoint paths Multi-way cut and labeling network design, facility location Relation to Games – local search  Price of anarchy – primal dual  Cost sharing

30. FOCS 200230 Metric Facility Location F is a set of facilities (servers). D is a set of clients. c ij is the distance between any i and j in D  F. Facility i in F has cost f i. client facility 5 4 2 3

31. FOCS 200231 Problem Statement We need to: 1) Pick a set S of facilities to open. 2) Assign every client to an open facility (a facility in S). Goal: Minimize cost of S +  p dist(p, S ). client facility 5 4 2 3 opened facility

32. FOCS 200232 What is known? All techniques can be used: Clever greedy [ Jain, Mahdian, Saberi ’02] Local search [starting with Korupolu, Plaxton, and Rajaraman ’98], can handle capacities LP and rounding: [starting with Shmoys, T, Aardal ’97] Here: primal-dual [starting with Jain-Vazirani’99]

33. FOCS 200233 What is the primal-dual method? Uses economic intuition from cost sharing –For each requirement, like  a  Lv x va = 1, someone has to pay to make it true… Uses ideas from linear programming: –dual LP and weak duality –But does not solve linear programs

34. FOCS 200234 Dual Problem: Collect Fees Client p has a fee α p (cost-share) Goal: collect as much as possible max  p α p Fairness: Do no overcharge: for any subset A of clients and any possible facility i we must have  p  A [α p – dist(p,i)]  f i amount client p would contribute to building facility i.

35. FOCS 200235 Exact cost-sharing All clients connected to a facility Cost share α p covers connection costs for each client p Costs are “fair” Cost f i of selecting a facility i is covered by clients using it  p α p = f(S)+  p dist(p,S), and both facilities are fees are optimal

36. FOCS 200236 Approximate cost-sharing Idea 1: each client starts unconnected, and with fee α p =0 Then it starts raising what it is willing to pay to get connected Raise all shares evenly α Example: = client = possible facility with its cost 44 4

37. FOCS 200237 Primal-Dual Algorithm (1) Each client raises his fee α evenly what it is willing to pay α = 1 Its α = 1 share could be used towards building a connection to either facility 44

38. FOCS 200238 Primal-Dual Algorithm (2) Each client raises evenly what it is willing to pay Starts contributing towards facility cost α = 2 44

39. FOCS 200239 Primal-Dual Algorithm (3) Each client raises evenly what it is willing to pay Three clients contributing α = 3 44

40. FOCS 200240 Primal-Dual Algorithm (4) Open facility, when cost is covered by contributions 4 clients connected to open facility Open facility α = 3 4

41. FOCS 200241 Primal-Dual Algorithm: Trouble Trouble: –one client p connected to facility i, but contributes to also to facility j 4 Open facility α = 3 4 ij p

42. FOCS 200242 Primal-Dual Algorithm (5) Close facility j: will not open this facility. Will this cause trouble? Client p is close to both i and j  facilities i and j are at most 2α from each other. 4 Open facility α = 3 4 ij p ghost

43. FOCS 200243 4 Primal-Dual Algorithm (6) Not yet connected clients raise their fee evenly Until all clients get connected 4 no not need to pay more than 3 Open facility α = 6 α =3 ghost

44. FOCS 200244 Feasibility + fairness ??  All clients connected to a facility  Cost share α p covers connection costs of client p  Cost f i of opening a facility i is covered by clients connected to it ?? Are costs “fair” ??

45. FOCS 200245 a set of clients A, and any possible facility i we have  p  A [α p – dist(p,i)]  f i –Why? we open facility i if there is enough contribution, and do not raise fees any further But closed facilities are ignored! and may violate fairness Are costs “fair”?? 44 open facility closed facility, ignored

46. FOCS 200246 Fair till it reaches a “ghost” facility. Let α’ q  α q be the fee till a ghost facility is reached Are costs “fair”?? 44 open facility Closed facility, ignored cause of closing j i p α’ q =4

47. FOCS 200247 Feasibility + fairness ??  All clients connected to a facility  Cost share α p covers connection costs for client p  Cost α p also covers cost of selected a facilities  Costs α’ p are “fair” How much smaller is α’  α ?? 44 p

48. FOCS 200248 How much smaller is α’  α ? q client met ghost facility j j became a ghost due to client p q i p stopped raising its share first  α p  α’ q  α q Recall dist(i,j)  2 α p, so α q  α’ q +2 α p  3α’ q 44 p j

49. FOCS 200249 Primal-dual approximation The algorithm is a 3-approximation algorithm for the facility location problem [Jain-Vazirani’99, Mettu-Plaxton’00] Proof: Fairness of the α’ p fees   p α’ p  min cost [max  min ] cost-recovery: f(S) +  p dist(p,S) =  p α p α  3α’ q  3-approximation algorithm

50. FOCS 200250 Outline of talk Techniques: Greedy Local search LP techniques: rounding Primal-dual Problems: Disjoint paths Multi-way cut and labeling network design, facility location Relation to Games – primal dual  Cost sharing – local search  Price of anarchy

51. FOCS 200251 primal dual  Cost sharing Dual variables α p are natural cost- shares: Recall: fair = no set is overcharged = core allocation  p  A α p – dist(p,i)  f i for all A and i. [Chardaire’98; Goemans-Skutella’00] strong connection between core cost- allocation and linear programming dual solutions See also Shapley’67, Bondareva’63 for other games

52. FOCS 200252 Primal-Dual  Cost-sharing Primal dual = for each requirement someone willing to pay to make it true Cost-sharing: only players can have shares. Not all requirements are naturally associated with individual players. Real players need to share the cost.

53. FOCS 200253 primal dual  Cost sharing Fair  no subset is overcharged Stronger desirable property: population monotone (cross- monotone): Extra clients do not increase cost-shares. Spanning-tree game: [Kent and Skorin-Kapov’96 and Jain Vazirani’01] Facility location, single source rent-or-buy [Pal-T’02]

54. FOCS 200254 Local search (for facility location) Local search: simple search steps to improve objective: add(s) adds new facility s delete(t) closes open facility t swap(s,t) replaces open facility s by a new facility t Key to approximation bound: How bad can be a local optima? 3-approximation [Charikar, Guha’00]

55. FOCS 200255 Local search  Price of anarchy in games Price of anarchy: facilities are operated by separate selfish agents Agents open/close facilities when it benefits their own objective. Agent’s “best response” dynamic: Simple local steps analogous to local search. Price of anarchy: How bad can be a stable state? 2-approximation in a related maximization game: [Vetta’02]

56. FOCS 200256 Conclusions for approximation Greedy, Local search clever greedy/local steps can lead to great results Primal-dual algorithms Elegant combinatorial methods Based on linear programming ideas, but fast, avoids explicitly solving large linear programs Linear programming very powerful tool, but slow to solve Interesting connections to issues in game theory