1. TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming

2. TU/e Algorithms (2IL15) – Lecture 12 2 Today: linear programming (LP)  most used and most widely studied optimization method  can be solved in polynomial time (input size measured in bits)  can be used to model many problems  also used in many approximation algorithms (integer LP + rounding) we will only have a very brief look at LP …  what is LP? what are integer LP and 0/1-LP ?  how can we model problems as an LP ? … and not study algorithms to solve LP’s

3. TU/e Algorithms (2IL15) – Lecture 12 3 Example problem: running a chocolate factory Assortment Cost and availability of ingredients cacaomilkhazelnutsretail price (100 g) Pure Black1001.99 Creamy Milk0.60.401.49 Hazelnut Delight 0.60.2 1.69 Super Nuts0.50.10.41.79 cost (kg)available (kg) cacao2.150 milk0.3550 hazelnuts1.930 How much should we produce of each product to maximize our profit ?

4. TU/e Algorithms (2IL15) – Lecture 12 4 Modeling the chocolate-factory problem cacaomilkhazelnutsprice (100 g) Pure Black1001.99 Creamy Milk0.60.401.49 Hazelnut Delight0.60.2 1.69 Super Nuts0.50.10.41.79 cost (kg)available (kg) cacao2.150 milk0.3550 hazelnuts1.930 variables we want to determine: production (kg) of the products b = production of Pure Black m = production of Creamy Milk h = production of Hazelnut Delight s = production of Super Nuts

5. TU/e Algorithms (2IL15) – Lecture 12 5 Modeling the chocolate-factory problem (cont’d) cacaomilkhazelnutsprice (100 g) Pure Black1001.99 Creamy Milk0.60.401.49 Hazelnut Delight0.60.2 1.69 Super Nuts0.50.10.41.79 cost (kg)available (kg) cacao2.150 milk0.3550 hazelnuts1.930 profits (per kg) of the products Pure Black19.9 – 2.1 = 17.8 Creamy Milk 14.9 – 0.6 x 2.1 + 0.4 x 0.35 = 13.5 Hazelnut Delight15.19 Super Nuts: 16.055 total profit: 17.8 b + 13.5 m + 15.19 h + 16.055 s

6. TU/e Algorithms (2IL15) – Lecture 12 6 Modeling the chocolate-factory problem cacaomilkhazelnutsprice (100 g) Pure Black1001.99 Creamy Milk0.60.401.49 Hazelnut Delight0.60.2 1.69 Super Nuts0.50.10.41.79 cost (kg)available (kg) cacao2.150 milk0.3550 hazelnuts1.930 we want to maximize the total profit 17.8 b + 13.5 m + 15.19 h + 16.055 s under the constraints b + 0.6 m + 0.6 h + 0.5 s ≤ 50 (cacao availability) 0.4 m + 0.2 h + 0.1 s ≤ 50 (milk availability) 0.2 h + 0.4 s ≤ 30 (hazelnut availability) This is a linear program: optimize linear function, under set of linear constraints

7. TU/e Algorithms (2IL15) – Lecture 12 7 1 −3 y ≥ − x + 3 y ≥ 2x − 4 y ≤ ½ x + 2 y ≥ 0 m constraints: linear function ≤ constant or ≥ or =, > and < not allowed n variables; here n=2, in chocolate example n=4, but often n is large objective function; must be linear function in the variables goal: maximize (or minimize) x y Linear programming Find values of real variables x, y such that  x − 3 y is maximized  subject to the constraints − 2 x + y ≥ − 4 x + y ≥ 3 − ½ x + y ≤ 2 y ≥ 0

8. TU/e Algorithms (2IL15) – Lecture 12 8 y = − x + 3 y = 2x − 4 y = ½ x + 2 1 −3 y = 0 Linear programming Find values of real variables x, y such that  x − 3 y is maximized  subject to the constraints − 2 x + y ≥ − 4 x + y ≥ 3 − ½ x + y ≤ 2 y ≥ 0 feasible region = region containing feasible solutions = region containing solutions satisfying all constraints feasible region is convex polytope in n-dim space

9. TU/e Algorithms (2IL15) – Lecture 12 9 Linear programming: Find values of real variables x 1, …, x n such that  given linear function c 1 x 1 + c 2 x 2 + … + c n x n is maximized (or: minimized)  and given linear constraints on the variables are satisfied constraints: equalities or inequalities using ≥ or ≤, cannot use Possible outcomes:  unique optimal solution: vertex of feasible region

10. TU/e Algorithms (2IL15) – Lecture 12 10 Linear programming: Find values of real variables x 1, …, x n such that  given linear function c 1 x 1 + c 2 x 2 + … + c n x n is maximized (or: minimized)  and given linear constraints on the variables are satisfied constraints: equalities or inequalities using ≥ or ≤, cannot use Possible outcomes:  unique optimal solution: vertex of feasible region  no solution: feasible region in empty

11. TU/e Algorithms (2IL15) – Lecture 12 11 Linear programming: Find values of real variables x 1, …, x n such that  given linear function c 1 x 1 + c 2 x 2 + … + c n x n is maximized (or: minimized)  and given linear constraints on the variables are satisfied constraints: equalities or inequalities using ≥ or ≤, cannot use Possible outcomes:  no solution: feasible region in empty  unique optimal solution: vertex of feasible region  bounded optimal solution, but not unique

12. TU/e Algorithms (2IL15) – Lecture 12 12 Linear programming: Find values of real variables x 1, …, x n such that  given linear function c 1 x 1 + c 2 x 2 + … + c n x n is maximized (or: minimized)  and given linear constraints on the variables are satisfied constraints: equalities or inequalities using ≥ or ≤, cannot use Possible outcomes:  no solution: feasible region in empty  unique optimal solution: vertex of feasible region  bounded optimal solution, but not unique  unbounded optimal solution

13. TU/e Algorithms (2IL15) – Lecture 12 13 n-dimensional vectors m x n matrix c, A, b are input, x must be computed Linear programming: standard form Maximize c 1 x 1 + c 2 x 2 + … + c n x n Subject to a 1,1 x 1 + a 1,2 x 2 + … + a 1,n x n ≤ b 1 a 2,1 x 1 + a 2,2 x 2 + … + a 2,n x n ≤ b 2.... a m,1 x 1 + a m,2 x 2 + … + a m,n x n ≤ b m x 1 ≥ 0 x 2 ≥ 0 … x n ≥ 0 Maximize c∙x subject to A x ≤ b and non-negativity constraints on all x i non-negativity constraints for each variable only “≤” (no “=“ and no “≥”) not: minimize

14. TU/e Algorithms (2IL15) – Lecture 12 14 Lemma: Any LP with n variables and m constraints can be rewritten as an equivalent LP in standard form with 2n variables and 2n+2m constraints. Proof. LP may not be in standard form because  minimization instead of maximization − negate objective function: minimize 2 x 1 − x 2 + 4x 3  maximize −2 x 1 + x 2 − 4 x n  some constraints are ≥ or = instead of ≤ − getting rid of =: replace 3 x + x 2 − x 3 = 5 by3 x + x 2 − x 3 ≤ 5 3 x + x 2 − x 3 ≥ 5 − changing ≥ to ≤: negate constraint 3 x + x 2 − x 3 ≥ 5  − 3 x − x 2 + x 3 ≤ − 5

15. TU/e Algorithms (2IL15) – Lecture 12 15 Lemma: Any LP with n variables and m constraints can be rewritten as an equivalent LP in standard form with 2n variables and 2n+2m constraints. Proof (cont’d). LP may not be in standard form because  minimization instead of maximization  some constraints are ≥ or = instead of ≤  variables without non-negativity constraint − for each such variable x i introduce two new variables u i and v i − replace each occurrence of x i by (u i − v i ) − add non-negativity constraints u i ≥ 0 and v i ≥ 0

16. TU/e Algorithms (2IL15) – Lecture 12 16 Lemma: Any LP with n variables and m constraints can be rewritten as an equivalent LP in standard form with 2n variables and 2n+2m constraints. Proof (cont’d).  variables without non-negativity constraint − for each such variable x i introduce two new variables u i and v i − replace each occurrence of x i by (u i − v i ) − add non-negativity constraints u i ≥ 0 and v i ≥ 0 new problem is equivalent to original problem : for any original solution there is new solution with same value − if x i ≥ 0 then u i = x i and v i = 0, otherwise u i = 0 and v i = − x i and vice versa − set x i = (u i − v i )

17. TU/e Algorithms (2IL15) – Lecture 12 17 Lemma: Any LP with n variables and m constraints can be rewritten as an equivalent LP in standard form with 2n variables and 2n+2m constraints. Instead of standard form, we can also get so-called slack form: – non-negativity constraint for each variable – all other constraints are =, not ≥ or ≤  Standard form (or slack form): convenient for developing LP algorithms  When modeling a problem: just use general form

18. TU/e Algorithms (2IL15) – Lecture 12 18 Algorithms for solving LP’s simplex method − worst-case running time is exponential − fast in practice interior-point methods − worst-case running time is polynomial in input size in bits − some are slow in practice, others are competitive with simplex method LP when dimension (=number of variables) is constant − can be solved in linear time (see course Geometric Algorithms)

19. TU/e Algorithms (2IL15) – Lecture 12 19 Modeling a problem as an LP  decide what the variables are (what are the choices to be made?)  write the objective function to be optimized (should be linear)  write the constraints on the variables (should be linear)

20. TU/e Algorithms (2IL15) – Lecture 12 20 Example: Max Flow

21. TU/e Algorithms (2IL15) – Lecture 12 21 Flow: function f : V x V → R satisfying  capacity constraint: 0 ≤ f (u,v) ≤ c(u,v ) for all nodes u,v  flow conservation: for all nodes u ≠ s, t we have flow in = flow out: ∑ v in V f (v,u) = ∑ v in V f (u,v) value of flow: |f | = ∑ v in V f (s,v) − ∑ v in V f (v,s) source sink 10 2 5 3 2 51 3 2 s t 3 2 / 3 / 4 / 2 / 1 / 2 / 1 / 0 / flow = 1, capacity = 5

22. TU/e Algorithms (2IL15) – Lecture 12 22 Modeling Max Flow as an LP  decide what the variables are (what are the choices to be made?) for each edge (u,v) introduce variable x uv ( x uv represents f(u,v) )  write the objective function to be optimized (should be linear) maximize ∑ v in V x sv − ∑ v in V x vs (note: linear function)  write the constraints on the variables (should be linear) x uv ≥ 0 for all pairs of nodes u,v x uv ≤ c(u,v) for all pairs of nodes u,v ∑ v in V x vu − ∑ v in V x uv = 0 for all nodes u ≠ s, t (note: linear functions)

23. TU/e Algorithms (2IL15) – Lecture 12 23 Modeling Max Flow as an LP Now write it down nicely maximize ∑ v in V x sv − ∑ v in V x vs subject to x uv ≥ 0 for all pairs of nodes u,v x uv ≤ c (u,v) for all pairs of nodes u,v ∑ v in V x vu − ∑ v in V x uv = 0 for all nodes u ≠ s, t Conclusion: Max Flow can trivially be written as an LP (but dedicated max-flow algorithm are faster than using LP algorithms)

24. TU/e Algorithms (2IL15) – Lecture 12 24 Example: Vertex Cover

25. TU/e Algorithms (2IL15) – Lecture 12 25 G = (V,E) is undirected graph vertex cover in G: subset C V such that for each edge (u,v) in E we have u in C or v in C (or both) Vertex Cover (optimization version) Input: undirected graph G = (V,E) Problem: compute vertex cover for G with minimum number of vertices  Vertex Cover is NP-hard.  there is a 2-approximation algorithm running in linear time. ∩

26. TU/e Algorithms (2IL15) – Lecture 12 26 Modeling Vertex Cover as an LP  decide what the variables are (what are the choices to be made?) for vertex v introduce variable x v ( idea: x v = 1 if v in cover, x v = 0 if v not in cover)  write the objective function to be optimized (should be linear) minimize ∑ v in V x v (note: linear function)  write the constraints on the variables (should be linear) −for each edge (u,v) write constraint x u + x v ≥ 1 (NB: linear function) −for each vertex v write constraint x u in {0,1} not a linear constraint

27. TU/e Algorithms (2IL15) – Lecture 12 27 integrality constraint: “x i must be integral” 0/1-constraint: “x i must be 0 or 1” integer LP: LP where all variables have integrality constraint 0/1-LP: LP where all variables have 0/1-constraint (of course there are also mixed versions)

28. TU/e Algorithms (2IL15) – Lecture 12 28 Theorem: 0/1-LP is NP-hard. Proof. Consider decision problem: is there feasible solution to given 0/1-LP? Which problem do we use in reduction? Need to transform 3-SAT formula into instance of 0/1-LP maximize y 1 (not relevant for decision problem, pick arbitrary function) subject to y 1 + y 2 + (1−y 3 ) ≥ 1 y 2 + (1−y 4 ) + (1−y 5 ) ≥ 1 (1 − y 2 ) + y 3 + y 5 ≥ 1 y i in {0,1} for all i ( x 1 V x 2 V ¬x 3 ) Λ (x 2 V ¬x 4 V ¬x 5 ) Λ (¬x 2 V x 3 V x 5 ) already saw reduction from Vertex Cover; let’s do another one: 3-SAT variable y i for each Boolean x i y i = 1 if x i = TRUE and y i = 0 if x i = FALSE

29. TU/e Algorithms (2IL15) – Lecture 12 29 Theorem: 0/1-LP is NP-hard. problem can be modeled as “normal” LP  problem can be solved using LP algorithms  problem can be solved efficiently problem can be modeled as integer LP (or 0/1-LP)  problem can be solved using integer LP (or 0/1-LP) algorithms  does not mean that problem can be solved efficiently (sometimes can get approximation algorithms by relaxation and rounding see course Advanced Algorithms)  there are solvers (software) for integer LPs that in practice are quite efficient

30. TU/e Algorithms (2IL15) – Lecture 12 30 Summary  what is an LP? what are integer LP and 0/1-LP?  any LP can be written in standard form (or in slack form)  normal (that is, not integer) LP can be solved in polynomial time (with input size measured in bits)  integer LP and 0/1-LP are NP-hard  when modeling a problem as an LP −define variables and how they relate to the problem −describe objective function (should be linear) −describe constraints (should be linear, not allowed) −no need to use standard or slack form, just use general form