1 Outline relationship among topics secrets LP with upper bounds by Simplex method basic feasible solution (BFS) by Simplex method for bounded variables extended basic feasible solution (EBFS) optimality conditions for bounded variables ideas of the proof examples Example 1 for ideas but inexact Example 2 for the exact procedure

2 A Depot for Multiple Products multi-product by a fleet of trucks depot Possible Formulation: objective function common constraints, e.g., trucks, DC capacity, etc. network constraints for type-1 product.... non-negativity constraints

3 A General Type of Optimization Problems structure of many problems: network constraints: easy other constraints: hard making use of the easy constraints to solve the problems solution methods: large-scale optimization column generation, Lagrangian relaxation, Dantzig-Wolfe decomposition … basis: linear programming, network optimization (and also non-linear optimization, integer optimization, combinatorial optimization) objective function network constraints non-negativity constraints hard constraints

4 Relationship of Solution Techniques two directions of theoretical development for network programming from special structures of networks from linear programming ideal: understanding development in both directions linear prog. network prog. int. prog. non-linear prog. dynamic prog. …

5 Relationship of Solution Techniques minimum cost flow column generation, Dantzig- Wolfe decomposition Lagrangian relaxation network algorithms network simplex shortest-path algorithms simplex method revised simplex method non-linear optimization linear algebra

6 Our Topics simplex method for bounded variables linkage between LP and network simplex optimality conditions for minimum cost flow networks minimum cost algorithms standard, and successive shortest path equivalence among network and LP optimality conditions revised simplex column generation Dantzig-Wolfe decomposition Lagrangian relaxation It takes more than one semester to cover these topics in detail! We will only cover the ideas.

7 Secrets

8 The Most Beautiful …

linear algebra 9 Maybe the Most Beautiful of All… algebraic properties geometric properties matrix properties

10 LP with Upper Bounds

upper bounds: common in network problems, e.g., an arc with finite capacity quite some theory of network optimization being from LP 11 LP with Upper Bounds

incorporate the upper-bound constraints into the set of functional constraints and solve accordingly 12 To Solve LP with Upper Bounds

In the simplex method the lower bound constraints 0 x do not appear in A. Is it possible to work only with A even with upper-bound constraints? Yes. 13 To Solve LP with Upper Bounds

A m n, m n, of rank m basic feasible solution (BFS) x of LP, i.e., feasible: Ax b, 0 x basic non-basic variables: (at least) n-m variables = 0 basic variables: m non-negative variables with linearly independent columns 14 BFS for Standard LP

A m n, m n, of rank m extended basic feasible solution ( EBFS ) x of LP with bounded variables, i.e., feasible: Ax b, 0 x u basic solution non-basic variables: (at least) n-m variables = 0, or = their upper bounds Basic variables: m variables of the form 0 x i u i, with linearly independent columns 15 Extended Basic Feasible Solution of LP with Bounded Variables

Maximum Conditions: BFS x is maximal if 0 for all non-basic variable x j = 0 Minimum Conditions: BFS x is minimal if 0 for all non-basic variable x j = 0 intuition : increase of the objective function by unit increase in x j maximum condition: no good to increase non-basic x j minimum condition: no good to decrease non-basic x j 16 Optimality Conditions of Standard LP

Maximum Conditions: EBFS x is maximal if 0 for all non-basic variable x j = 0, and 0 for all non-basic variable x j = u j Minimum Conditions: EBFS x is minimal if 0 for all non-basic variable x j = 0, and 0 for all non-basic variable x j = u j 17 Optimality Conditions of LP with Bounded Variables

18 How to Prove?

optimality conditions of the EBFS from duality theory and complementary slackness conditions 19 General Idea

primal-dual pair Theorem 1 (Complementary Slackness Conditions) if x primal feasible and y dual feasible then x primal optimal and y dual optimal iff x j (y T A j c j ) = 0 for all j, and y i (b i A i x) = 0 for all i 20 Complementary Slackness Conditions

primal-dual pair Theorem 2 (Necessary and Sufficient Condition) if x primal feasible then x primal optimal iff there exists dual feasible y such that x and y satisfy the Complementary Slackness Conditions 21 Complementary Slackness Conditions

by Theorem 2, primal feasible x and dual feasible (y T, T ) are optimal iff x j (y T A j + j - c j­ ) = 0, j y i (b i - A i x) = 0, i j (u j - x j­ ) = 0, j 22 Complementary Slackness Conditions for LP with Bounded Variables

optimality conditions of the EBFS from duality theory and complementary slackness conditions ideas of the proof given an EBFS x satisfying the upper-bound optimality conditions then possible to find dual feasible variables (y T, T ) T such that x and (y T, T ) T satisfy the complementary slackness conditions 23 General Idea of the Proof

max 2x + 5y, min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y Example 1. Upper-Bound Constraints as Functional Constraints

25 Examples of LP with Bounded Variables

min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8. max. value = 44 x * = 2 and y * = 8 26 Example 1. Upper-Bound Constraints as Functional Constraints

27 The following procedure is not exactly the Simplex Method for Bounded Variables. It primarily brings out the ideas of the exact method.

y as the entering variable 2y + s 1 = 20 y + s 2 = 16 y 8 28 Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables -5 min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

mark the non-basic variable y at its upper bound for y = 8 obj. fun.: -2x – 5y – z = 0 -2x - z = 40 eqt. (1): x + 2y + s 1 = 20 x + s 1 = 4 eqt. (2): 2x + y + s 2 = 16 2x + s 2 = 8 29 Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables

x as the entering variable x + s 1 = 4 2x + s 2 = 8 x 2 30 Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

for x at its upper bound 2, mark x, and obj. fun.: -2x – z = 40 -z = 44 eqt. (1): x + s 1 = 4 s 1 = 2 eqt. (2): 2x + s 2 = 8 s 2 = 4 31 Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables min 2x 5y, s.t. x + 2y 20, 2x + y 16, 0 x 2, 0 y 8.

satisfying the optimality condition for bounded variables 0 for all non-basic variable x j = 0, and 0 for all non-basic variable x j = u j z * = -44, with x * = 2 and y * = 8 32 Example 1. Upper-Bound Constraints by Optimality Conditions of Bounded Variables

in general, variables swapping among all sorts of status non-basic at 0 basic at 0 basic between 0 and upper bound basic at upper bound non-basic at upper bound Simplex method for bounded variables: a special algorithm to record all possibilities 33 Example 1 Being Too Specific

34 The following example follows the exact procedure of the Simplex Method for Bounded Variables.

max 3x 1 + 5x 2 + 2x 3 min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x Example 2

potential entering variable: x 2 bounded by upper bound 3 define = u 2 -x 2 = 3-x 2 36 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.

37 Example 2 by Simplex Method for Bounded Variables

x 1 as the (potential) entering variable s 2 as the leaving variable a pivot operation as in standard Simplex Method 38 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.

which can be an entering variable? can s 1 be a leaving variable? Yes can x 1 be a leaving variable? Yes 39 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.

when = 1.25, x 1 reaches its upper bound 4 replace x 1 by and is a basic variable = 0 result 40 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.

. a normal pivot operation with a ij < 0 41 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.

minimum z * = , x 1 * = 4, x 2 * = 1.75, x 3 * = 0 42 Example 2 by Simplex Method for Bounded Variables min 3x 1 5x 2 2x 3, s.t. x 1 + x 2 + 2x 3 7, 2x 1 + 4x 2 + 3x 3 15, 0 x 1 4, 0 x 2 3, 0 x 3 3.