1 Introduction to Linear Programming

2 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. X1X2X3X4X1X2X3X4 Unit: thousand To win at least urban votes: -2x 1 +8x 2 +0x 3 +10x 4  50. To win at least suburban votes: 5x 1 +2x 2 +0x 3 +0x 4  100. To win at least rural votes: 3x 1 -5x 2 +10x 3 -2x 4  25. Want to minimize x 1 + x 2 + x 3 + x 4

3 Minimization linear program : minimize x 1 + x 2 + x 3 + x 4 subject to -2x 1 + 8x 2 + 0x x 4  50 5x 1 + 2x 2 + 0x 3 + 0x 4  100 3x 1 - 5x 2 +10x 3 - 2x 4  25 x 1, x 2, x 3, x 4  0. Maximization linear program : maximize x 1 + x 2 subject to 4x 1 - x 2  8 2x 1 + x 2  10 5x 1 - 2x 2  -2 x 1, x 2  0. Standard form, objective function, feasible solution and region, simplex algorithm, ellipsoid algorithm, interior-point method.

4 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

5 Standard and Slack forms: Standard form: maximize c j x j subject to a ij x j  b i for i=1, …, m. x j  0 for j=1, …, n.  Objective function  Constraints  Nonnegative constraints maximize c T x subject to A x  b x  0

6 Minimization linear program: minimize -2x x 2 subject to x 1 + x 2 = 7 x 1 - 2x 2  4 x 1  0. Maximization linear program: maximize 2x 1 -3 x 2 subject to x 1 + x 2 = 7 x 1 - 2x 2  4 x 1  0. Converting linear programs into standard form:

7 If x j has no nonnegative constraint, we replace x j by x’ j - x’’ j, where x’ j and x’’ j  0. maximize 2x x’ x’’ 2 subject to x 1 + x’ 2 - x’’ 2 = 7 x x’ x’’ 2  4 x 1, x’ 2, x’’ 2  0.

8 Since x = y iff x  y and y  x, we can replace equality constraint by the pair of inequality constraints. maximize 2x x x 3 subject to x 1 + x 2 - x 3  7 x 1 + x 2 - x 3  7 x x x 3  4 x 1, x 2, x 3  0.  - x 1 - x 2 + x 3  -7

9 Converting linear programs into slack form: maximize 2x x x 3 subject to x 4 = 7 - x 1 - x 2 + x 3 x 5 = -7 + x 1 + x 2 - x 3 x 6 = 4 - x x x 3 x 1, x 2, x 3, x 4, x 5, x 6  0. x 4, x 5, x 6 : basic variables x 1, x 2, x 3 : nonbasic variables z = 2x x x 3 x 4 = 7 - x 1 - x 2 + x 3 x 5 =-7 + x 1 + x 2 - x 3 x 6 = 4 - x x x 3 slack form:

10 N: the set of indices of nonbasic variables, |N|=n. B: the set of indices of basic variables, |B|=m. N  B ={1, 2, …, m+n}. A slack form: z = 28 - x 3 /6 - x 5 /6 - 2 x 6 /3 x 1 = 8 + x 3 /6 + x 5 /6 - x 6 /3 x 2 = 4 - 8x 3 /3 - 2 x 5 /3 + x 6 /3 x 4 = 18 - x 3 /2 + x 5 /2

11 Formulating problems as LP: Single-pair shortest-path: given G=(V, E), w, s and t, we want to compute d t, which is the weight of a shortest path from s to t. min d t subj. to d v ≤ d u + w(u, v) for each (u, v) in E, d s = 0. Maximum flow: max  v f sv -  v f vs subj. to f uv ≤ c(u, v) for each u, v in V, f uv ≥ 0 for each u, v in V,  v f uv =  v f vu for each u in V-{s, t}.

12 Minimum-cost flow:

13 Minimum cost flow: min  (u,v) a(u,v) f uv subj. to f uv  c(u, v) for each u, v in V, f uv ≥ 0 for each u, v in V,  v f vu -  v f uv = 0 for each u in V-{s, t},  v f sv -  v f vs = d,

14 Multi-commodity flow: Given a flow network and k different commodities, K 1, K 2,..., K k, where each K i = (s i, t i, d i ) and s i is the source of commodity i, t i is the sink and d i is the demand– the desired flow from s i to t i. Define a flow f i for commodity i and f uv =  i=1..k f iuv. Want to determine whether it is possible to find such a flow. LP is the only known polynomial-time algorithm for this problem. min 0 subj. to  i=1..k f iuv  c(u, v) for each u, v in V, f iuv ≥ 0 for each u, v in V and i= 1..k,  v f iuv -  v f ivu = 0 for each u in V-{s i, t i } and i= 1..k,  v f is i v -  v f ivs i = d i for each i = 1..k.

15 Simplex algorithm: An example: Max 3 x 1 + x 2 + 2x 3 Subject to x 1 + x x 3  30 2x 1 + 2x 2 + 5x 3  24 4x 1 + x 2 + 2x 3  36 x 1, x 2, x 3  0 Slack form: z = 3 x 1 + x 2 + 2x 3 x 4 = 30 - x 1 - x x 3 x 5 = x 1 - 2x 2 - 5x 3 x 6 = x 1 - x 2 - 2x 3 x 1, x 2, x 3, x 4, x 5, x 6  0 (0, 0, 0, 30, 24, 36) is a basic solution. Want to find a basic feasible solution (BFS) from basic solution. Goal: in each iteration, reformulate the LP so that BFS has a greater object value. Attempt: select a non-basic variable x e whose coefficient in the object function is positive and increase its value as much as possible without violating any constraint. Then x e becomes basic and some x l becomes non-basic.

16 Slack form: z = 3 x 1 + x 2 + 2x 3 x 4 = 30 - x 1 - x x 3 x 5 = x 1 - 2x 2 - 5x 3 x 6 = x 1 - x 2 - 2x 3 x 1 : entering variable, x 6 : leaving variable. z = 27+ x 2 /4 + x 3 /2 - 3 x 6 /4 x 1 = 9 - x 2 /4 - x 3 /2 - x 6 /4 x 4 = x 2 /4 - 5 x 3 /2 + x 6 /4 x 5 = x 2 /2 - 4x 3 + x 6 /2. pivoting z = 111/4+ x 2 /16 - x 5 / x 6 /16 x 1 = 33/4 - x 2 /16 + x 5 /8 - 5x 6 /16 x 3 = 3/2 - 3 x 2 /8 - x 5 /4 + x 6 /8 x 4 = 69/4 + 3 x 2 /16 +5x 5 /8 - x 6 /16. z = 28- x 3 /6 - x 5 /6 - 2 x 6 /3 x 1 = 8 + x 3 /6 + x 5 /6 - x 6 /3 x 2 = x 3 /3 - 2x 5 /3 + x 6 /3 x 4 = 18 - x 3 /2 +x 5 /2.

17

18 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

19

20 1.How do we determine if a linear program is feasible? 2.What do we do if the LP is feasible, but the initial basic solution is not feasible? 3. How do we determine if a linear program is unbounded? 4. How do we choose the entering and leaving variables?

21 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

22 Lemma 2: Given a linear program (A, b, c), suppose the call to INITIALIZE-SIMPLEX in line 1 of SIMPLEX returns a slack form for which the basic solution is feasible. Then if SIMPLEX returns a solution in line 16, that solution is a feasible solution to the LP. If SIMPLEX returns “unbounded” in line 10, the LP is unbounded.

23 Lemma 3: Let I be a set of indices. For each i in I, let  i and  i be real numbers and let x i be a real-valued variable. Let  be any real number. Suppose that for any settings of the x i we have:

24 Lemma 4: Let (A, b, c) be an LP in standard form. Given a set B of basic variables, the associated slack form is uniquely determined. Pf: By contradiction, suppose there are two slack form with the same B:

25 Lemma 5: If SIMPLEX fails to terminate in at most (m+n)!/m!n! iterations, then it cycles. Lemma 6: (Bland’s rule) In line 3 and 8 of SIMPLEX, ties are always broken by choosing the variable with the smallest index, then SIMPLEX must terminate.

26 Lemma 7: Assuming that INITIALIZE-SIMPLEX returns a slack form for which the basic solution is feasible, SIMPLEX either reports that an LP is unbounded, or it terminates with a feasible solution in at most (m+n)!/m!n! iterations.

27 Duality: primal and dual max c j x j subject to a ij x j  b i for i=1, …, m. x j  0 for j=1, …, n. Max 3 x 1 + x 2 + 2x 3 Subject to x 1 + x x 3  30 2x 1 + 2x 2 + 5x 3  24 4x 1 + x 2 + 2x 3  36 x 1, x 2, x 3  0 Min 30 y y y 3 Subject to y 1 + 2y y 3  3 y 1 + 2y 2 + y 3  1 3y 1 + 5y y 3  2 y 1, y 2, y 3  0

28

29

30

31 Finding an initial solution: Max 2 x 1 - x 2 Subject to 2x 1 - x 2  2 x 1 - 5x 2  - 4 x 1, x 2  0 x 1 =0, x 2 =0 is NOT a feasible solution. L aux : Max -x 0 Subject to 2x 1 - x 2 - x 0  2 x 1 - 5x 2 - x 0  - 4 x 1, x 2, x 0  0 Slack form: z = - x 0 x 3 = 2 - 2x 1 + x 2 + x 0 x 4 = -4 - x 1 +5x 2 + x 0 z = -4 - x 1 +5 x 2 - x 4 x 0 = 4 + x 1 -5 x 2 + x 4 x 3 = 6 - x 1 -4 x 2 + x 4

32 z = -4 - x 1 +5 x 2 - x 4 x 0 = 4+ x 1 -5 x 2 + x 4 x 3 = 6- x 1 -4 x 2 + x 4 (4,0,0,6,0) is a BFS. z = - x 0 x 2 = 4/5- x 0 /5 + x 1 /5 + x 4 /5 x 3 = 14/5+ 4x 0 /5 - 9x 1 /5 + x 4 /5 The following has a BFS. z = -4/5 +9x 1 /5 - x 4 /5 x 2 = 4/5 + x 1 /5 + x 4 /5 x 3 = 14/5- 9x 1 /5 + x 4 /5 Take x 0 = 0. Then x 2 = 4/5 + x 1 /5 + x 4 /5, and 2x 1 - x 2 = -4/5 +9x 1 /5 - x 4 /5.

33 Lemma 11: Let L be a LP in standard form. Let L aux be the following LP with n+1 variables: maximize - x 0 subject to a ij x j - x 0  b i for i=1, …, m. x j  0 for j = 0, …, n. Then L is feasible iff the optimal object value of L aux is 0.

34 Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

35 Lemma 12: If a LP L has no feasible solution, INITIALIZE-SIMPLEX returns “infeasible.” Otherwise, it returns a valid slack form for which the basic solution is feasible. Theorem 13: (Fundamental theorem of LP) Any linear program L, given in standard form, either 1.has an optimal solution with a finite objective value, 2.is infeasible, or 3.is unbounded. If L is infeasible, SIMPLEX returns “ infeasible.” If L is unbounded, SIMPLEX returns “unbounded.” Otherwise, SIMPLEX returns an optimal solution with a finite objective value.