1. Instructor Neelima Gupta ngupta@cs.du.ac.in

3. Table of Contents Lp –rounding Dual Fitting LP-Duality

4. Linear Programming Problem A linear programming (LP) problem is an optimization problem in which we minimize or maximize a linear objective function subject to a given set of linear constraints. Example: Minimize 3x1 − 5x2 + + 3x3 + 2x4 subject to: 3x1 + 4x2 = 6 −x3 + 2x1 − x2 ≥ 22 x5 ≤ −3.5 x3 +.5x4 =.8 xi ≥ 0 for all i

5. Feasible Solution A feasible solution to a linear program is a solution that satisfies all constraints. Optimal Solution An optimal solution to a linear program is a feasible solution with the largest(smallest) objective function value for a maximization(minimization) problem. Solutions Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

6. Many optimization problems involve selecting a subset of a given set of elements. Examples: A vertex cover is a subset of vertices. A spanning tree is really a subset of edges. A knapsack solution is a subset of items. Can be formulated as LPs with integrality constraints.

7. Integer Program An Integer Program (IP) is an LP with Integrality Constraints Integrality Constraints: Some or all the variables are constrained to be integers.

8. LPs can be solved efficiently (polynomially but slowly). IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral. Solving Linear/Integer Programming Problems Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

9. Using Indicator Variables Many selection problems can be formulated as IPs using indicator variables (or 0-1 variables). An indicator variable is defined for each element. A value of 1 indicating the selection of the element and a value of 0 indicating otherwise.

10.  vertex cover  Set Cover  Knapsack Few Examples are : Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

11. Example: Unweighted Vertex Cover Variables: {x v | v ∈ V }. The IP: Minimize ∑ x v s.t. x u + x v ≥ 1 ∀ (u, v) ∈ E, x v ∈ {0, 1} ∀ v ∈ V.

12. Example: Knapsack Let the item names be {1,..., n}. Variables: {x i | 1 ≤ i ≤ n}. The IP: Minimize ∑ i c i x i s.t. ∑ i s i x i ≤ K, x i ∈ {0, 1} ∀ 1 ≤ i ≤ n.

13. Solving Linear/Integer Programming Problems LPs can be solved efficiently (polynomially but slowly). IPs generally cannot be solved efficiently (it is NP hard). Some specific IPs can be solved efficiently. Actually, their LP optimal is guaranteed to be integral.

14. LP Relaxation (Drop the integrality constraint) Example: Unweighted Vertex Cover The IP: Minimize ∑ v x v s.t. x u + x v ≥ 1 ∀ (u, v) ∈ E, x v ∈ {0, 1} ∀ v ∈ V. The LP relaxation: Minimize ∑ v x v s.t. x u + x v ≥ 1 ∀ (u, v) ∈ E, x v >= 0 ∀ v ∈ V.

15. Variables: {x v | v ∈ V }. The IP: Min ∑C v x v where C v : cost associated with vertex x v : indicator variable s.t: x u + x v ≥ 1 ∀ (u, v) ∈ E x v ∈ {0, 1} ∀ v ∈ V Example: Weighted Vertex Cover Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

16. Example: Weighted Vertex Cover The IP: Min ∑C v x v s.t: x u + x v ≥ 1 ∀ (u, v) ∈ E x v ∈ {0, 1} ∀ v ∈ V The LP relaxation: Min ∑C v x v s.t: x u + x v ≥ 1 ∀ (u, v) ∈ E x v ≥ 0 ∀ v ∈ V LP Relaxation (Drop the integrality constraint) Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

18. If x v ≥ ½, round it up to 1 Else round it down to 0. Here x v is the solution obtained from LP E.g: LP: ¼ c 1 + ½ c 2 + ¾ c 3 + 4∕5 c 4 IP : c 2 + c 3 + c 4 LP rounding Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

19. Let (u,v) ∈ E Since the solution of LP is feasible, values of x v, v ∈ V, satisfy x u + x v ≥ 1(1) ⇒ atleast one of x u and x v ≥ ½ Assume x’ u and x’ v be the solutions obtained after rounding, then at least one of them must be 1, i.e. x’ u + x’ v ≥ 1 So the solution, obtained after rounding, is feasible. Claim 1: Solution Obtained is feasible Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

20. According to the strategy some of the variables have been increased to a maximum of double & some have been reduced to 0, i.e C v’ <= 2C v. Claim 2: C(S) ≤ 2L OPT Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

21. So, C(S): cost of solution obtained by IP C(S) ≤ ∑ v’ C v’ x v’ ≤ 2 ∑ v C v x v ( x’ v ≤ 2* X v ) = 2 LP OPT Hence claim 2 follows Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

22. A finite set (universe) U of n elements, U= {e 1, e 2,…, e n }, a collection of subsets of U i.e. S 1, S 2, …., S k with some cost, select a minimum cost collection of these sets that covers all elements of U. Set Cover Problem Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

23. IP: IP: Indicator variable x s, x s ∈ {0,1} x s =0 if set S is not picked x s =1 if set S is picked Min ∑ s C s x s s.t. ∑ s:e belongs to S x s ≥ 1 ∀ e ∈ U x s = {0,1} LP Relaxation: Min ∑ s C s x s s.t. ∑ s:e belongs to S x s ≥ 1 ∀ e ∈ U x s > 0 Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

24. Let f denote the maximum frequency of any element in U S i  Find an optimal solution to LP-Relaxation x s >1/f round it to 1 x s <1/f discard the set, i.e. round it down to 0. LP rounding for SC Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

25. Claim 1: solution is feasible Claim 2: It gives factor f approximation Claims Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

26. Let, e i ∈ U, 1≤i≤n S be the collection of subsets of U e m : 1<m <n belongs to l subsets of S where 1<l<k Since the solution of LP is feasible i.e. values of x s s ∈ S obtained satisfies x s 1 + x s 2 + x s 3 + ….+ x s l >1(1) ⇒ atleast one of x s1, x s2, x s3,…., x l >1/f ⇒ x’ s1 + x’ s2 + x’ s3 +….+ x’ l > 1 Where x’ si is the solution obtained after rounding. Thus it is feasible. Claim 1: Solution is feasible Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

27. For each set s ∈ Collection of picked sets(S), x s has been increased by a factor of atmost f. Let C(s): Cost of our solution Therefore, C(S) ≤ ∑ s C s x’ s ∀ s ∈ S ≤ f ∑ s C s x s ( x’ s ≤ f* x s ) = f LP OPT Hence it is a factor ’f’ approximation. Note: f factor could be large. Later we’ll see a technique of rounding that gives O(log n) factor. Claim 2: Factor f approximation Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)

29. Linear Programming - Example Minimize 8x 1 + 5x 2 + 5x 3 + 2x 4 subject to: 3x 1 + 4x 2 ≥ 6 3x 2 + x 3 + x 4 ≥ 5 x i ≥ 0 for all i x = (2, 1,0, 3) is a feasible solution. 8*2 + 5*1 + 2*3 = 27 is an upper bound.

30. What is the Lower Bound? Minimize 8x 1 + 5x 2 + 5x 3 + 2x 4 subject to: 3x 1 + 4x 2 ≥ 6 3x 2 + x 3 + x 4 ≥ 5 x i ≥ 0 for all i LB: 8x 1 + 5x 2 + 5x 3 + 2x 4 ≥ 3x 1 + 4x 2 ≥ 6 Better LB: 8x 1 + 5x 2 + 5x 3 + 2x 4 ≥ (3x 1 + 4x 2 ) + (3x 2 + x 3 + x 4 ) ≥ 6+5 = 11

31. How to compute a good LB Minimize 8x 1 + 5x 2 + 5x 3 + 2x 4 subject to: 3x 1 + 4x 2 ≥ 6 ……………….y 1 3x 2 + x 3 + x 4 ≥ 5……………y 2 x i ≥ 0 for all i Assign a non-negative coefficient y i to every inequality such that 8x 1 + 5x 2 + 5x 3 + 2x 4 ≥ y 1 (3x 1 + 4x 2 ) + y 2 (3x 2 + x 3 + x 4 ) Then, LHS ≥ 6y 1 + 5y 2. We are interested in finding y i ’s such that RHS is maximum. This leads to our dual problem.

32. The corresponding dual for the given example will be: max 6y 1 + 5y 2 such that, 3y 1 < 8 4y 1 + 3y 2 < 5 y 1 < 5 y 2 < 2 and, y i > 0for all i Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

33. Weak Duality Theorem Theorem: If x and y are feasible then, > Proof: > = > Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

34. Set Cover x s is 1 iff set S in included in the cover. The Primal : Objective : min ∑ C s x s s.t > 1 U x s = {0,1} LP relaxation: x s > 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

35. Introduce an indicator variable y e for each of the constraints in primal. The Dual : objective: max s.t < C S i for i = 1 to k Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

36. Example S = { x, y, z, w} S 1 = { x, y} S 2 = { y, z} S 3 = { x, w, y} Let x s 1, x s 2, x s 3 be an indicator variable for S 1, S 2, S 3 respectively. Let C s 1, C s 2, C s 3 is the cost of S 1, S 2, S 3 respectively. Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

37. Primal Min : C s 1 x s 1 + C s 2 x 2 + C s 3 x 3 Subject to x s 1 + x s 3 > 1 (y x ) x s 1 + x s 2 + x s 3 > 1 (y y ) x s 2 > 1 (y z ) x s 3 > 1 (y w ) x s 1, x s 2, x s 3 > 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

38. Dual Max: y x + y y + y z + y w Subject to y x + y y < C s 1 y y + y z < C s 2 y x + y y + y w < C s 3 y x, y y, y z, y w > 0 Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)

39. From set cover via lp

41. Complementary Slackness Conditions

42. Relaxed Complementary Slackness Conditions

43. Example: Weighted Vertex Cover Primal: Min ∑C v x v s.t: x u + x v ≥ 1 ∀ (u, v) ∈ E x v ∈ {0, 1} ∀ v ∈ V Dual: Max ∑y e s.t: ∑ e:e is incident on v y e < C v ∀ v ∈ V y e ∈ {0, 1} ∀ e ∈ E

44. Primal Dual Schema 1 U = empty, y = 0 For each edge e = (u, v) ye = min {c(u) − ∑ e ′ :u ∈ e ′ y e ′, c(v) − ∑ e ′ :v ∈ e ′ y e ′ } U = U union argmin {c(u) − ∑ e ′ :u ∈ e ′ y e ′, c(v) − ∑ e ′ :v ∈ e ′ y e ′ } Output U

45. 5 4 3 2 3 1 3 2 7 Thanks to Neha& Neha Katyal

46. 5 4 (1) 3 (0) Y e =3 For every edge pick minimum of two vertices Min{4,3} = 3 Set ye=3 U has vertex having red color 1 2 3 3 2 7 Thanks to Neha& Neha Katyal

47. 5(4) 4 (1) (0) 3(0) Y e =1 Y e =3 Min{1,5} = 1 Set ye=1 1 2 3 3 2 7 Thanks to Neha& Neha Katyal

48. 5(4) 4 (1) (0) 3(0) Y e =1 Y e =3 Min{1,0} = 0 Set ye=0 Min{2,0} = 0 Set ye=0 Min{3,0} = 0 Set ye=0 Y e =0 1 2 3 3 2 7 Thanks to Neha& Neha Katyal

49. 5(4)(0) Y e =4 7(3) 4 (1) (0) 3(0) Y e =1 Y e =3 Y e =0 3 2 Thanks to Neha& Neha Katyal

50. Y e =0 3 4 (1) (0) 3(0) Y e =1 Y e =3 Y e =0 5(4)(0) Y e =4 7(3) 2 Thanks to Neha& Neha Katyal

51. 2 (0) 3 (1) Y e =2 4 (1) (0) 3(0) Y e =1 Y e =3 Y e =0 5(4)(0) Y e =4 7(3) Y e =0 Red-colored nodes form a vertex-cover Thanks to Neha& Neha Katyal

52. 2 (0) 3 (1) Y e =2 4 (1) (0) 3(0) Y e =1 Y e =3 Y e =0 5(4)(0) Y e =4 7(3) Y e =0 Red-colored nodes form a vertex-cover Thanks to Neha& Neha Katyal

53. Solution is feasible Trivial, since the algorithm runs for every edge. Let e= (u,v) be an edge. Suppose if possible, none of the x u and x v has been set to 1 i.e constraints corresponding to u and v have not yet gone tight and we have a y e that can be raised. That means the algorithm has not yet completed.

54. Solution is 2 factor For every x v > 0, dual constraint is tight (trivially). For every edge e = (u,v), 1 < x u + x v < 2 Hence, by relaxed CSC, cost of the solution is at most twice the OPT.

55. Primal-Dual Schema 2 (Ignore) Raise the dual variables uniformly until one or more of the constraints become tight. Freeze the dual variables contributing to these constraints. Set the corresponding primal variable to 1. If more than one constraint becomes tight, take them one by one in an arbitrary order.

56. 5 4 3 2 3 1 3 2 7 Thanks to Neha& Neha Katyal

57. 5 4 3 (1) Y e =3/4 1 2 3 3 2 7

58. 5 4 3 (1) Y e =3/4 1 2 3 3 2 7 Y e =3/2

59. 5 4 3 (1) Y e =3/4 1 2 3 3 2 7 Thanks to Neha& Neha Katyal Y e =3/4 Y e =3/2 Y e =7/4 Y e =3/2

60. Solution is feasible Let e= (u,v) be an edge. Suppose if possible, none of the x u and x v has been set to 1 i.e consraints corresponding to u and v have not yet gone tight and we have a y e that can be raised. That means the algorithm has not yet completed.

61. Solution is 2 factor For every x v > 0, dual constraint is tight (trivially). For every edge e = (u,v), 1 < x u + x v < 2 Hence, by relaxed CSC, cost of the solution is at most twice the OPT.