IEOR 4004 Midterm Review (part I) March 10, 2014

Summary Modeling optimization problems Linear program Simplex method Algorithm Initialization Variants Sensitivity analysis (in part II) Duality (in part II) Upper bounds and Shadow prices Complementary slackness

Mathematical modeling Simplified (idealized) formulation Limitations Only as good as our assumptions/input data Cannot make predictions beyond the assumptions We need more maps

Mathematical modeling Problem simplification Model formulation Model Algorithm selection Numeric calculation Problem Solution Interpretation Sensitivity analysis

Mathematical modeling Deterministic = values known with certainty Stochastic = involves chance, uncertainty Linear, non-linear, convex, semi-definite

Formulating an optimization problem Linear program Linear objective Decision variables Objective Constraints Domains x1, x2, x3, x4 + x1 (1 − 2x2)2 Minimize 2x2 + 3x3 (x1)2 + (x2)2 + (x3)2 + (x4)2 ≤ 2 x1 − 2x2 + x3 − 3x4 ≤ 1 − x1 + 3x2 + 2x3 + x4 ≥ − 2 − 2x1x3 = 1 Linear constraints x1 ≥ 0 x3 in {0,1} x4 in [0,1] x2 ≠ 0.5 Sign restriction

Linear program solution = values of variables that together satisfy all constraints Feasible solution = satisfies also sign constraints Infeasible solution = not feasible Basis = set of m variables (where m = # of equations) Variable in basis = basic variable, all other non-basic Basic solution = set all non-basic variables to zero {x1, x2} x1 − 2x2 + x3 − 3x4 = 1 − x1 + 3x2 + 2x3 + x4 = − 2 x1 = − 1 x2 = −1 x3 = 0 x4 = 0 x1 = 4/3 x2 = 0 x3 = −1/3 x4 = 0 x1 ≥ 0

Linear program Feasible region = set of all feasible solutions LP is Infeasible if feasible region empty LP is Unbounded if the objective function is unbounded over the feasible region objective function grows beyond bounds objective function growth is limited

Simplex algorithm Converting to standard form Add slack variables Substitute negative variables Substitute unrestricted variables Dictionary x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4− x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4− − 3x4+ + 3x4− + x4+ − x4− x1 − 2x2 − x1 + 3x2 − x3’ − 2x3’ + x3 + 2x3 + x5 = − x6 = − 3x4 + x4 ≤ ≥ 3 − 2 z = − 2x2 + 3x3’ x1, x2 ≥ 0 x5, x6 ≥ 0 x3 ≤ 0 x3’ ≥ 0 x4+, x4− ≥ 0

Simplex method (maximization) START x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4− x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4− Initial feasible solution z = − 2x2 + 3x3’ xj Pivot xj to the basis Optimality test Are all coefficients in z non-positive ? Ratio test Find min b/a a = coeff of xi in xj b = value of xj a, b opposite signs Variable selection xi NO YES END Alternative solutions? optimal solution found xj does not exist LP is unbounded

Simplex algorithm Anatomy of a dictionary Ratio test: x5: 3/1 = 3 x6: no constraint (different sign) (same sign) x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4− x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4− basis objective(s) z = − 2x2 + 3x3’ zero coeff negative coeff positive coeff x3’ enters x5 leaves (min ratio)

Simplex/Graphical method x3, x4 are slack variables Illustration profit: 180 150 80 max 3x1 +2x2 x1 + x2 + x3 = 80 2x1 + x2 + x4 = 100 x1, x2, x3, x4 ≥ 0 max 3x1 +2x2 x1 + x2 ≤ 80 2x1 + x2 ≤ 100 x1, x2 ≥ 0 z start: x1=0, x2=0, x3=80, x4=100 x1 increases until x4=0 60 x2 increases until x3=0 optimal 40 x3 x1 x4 isoprofit line 20 x2 3x1 +2x2 = profit 20 40 60 80

Initialization - Phase I Goal: make all original variables non-basic (=0) If all-0 doesn’t satisfy a constraint, add artificial variable ai: add +ai if the right-hand side is > 0 add –ai if the right-hand side < 0 Add slack variables (where needed) New objective: minimize the sum of artificial variables Starting basis: all artificial + slack variables from equations without artificial variables If optimal value negative, then LP Infeasible Else drop artificial variables and introduce z Maximize w = − a2 − a3 z = 2x2 − 3x3 s1 = 1 − x1 + 2x2 − x3 + 3x4 a2 = 2 − x1 + 3x2 + 2x3 + x4 a3 = 1 − x2 − x3 − 2x4 + e3 x1 = 5/2 − (7/2)x3 + (1/4)a2 − (7/4)a3 − (5/4)s1 + (7/4)e3 x2 = 0 − 2x3 + (1/2)a2 − (1/2)a3 − (1/2)s1 + (1/2)e3 x4 = 1/2 + (1/2)x3 − (1/4) a2 − (1/4)a3 + (1/4)s1 + (1/4)e3 x1 = 5/2 − (7/2)x3 − (5/4)s1 + (7/4)e3 x2 = 0 − 2x3 − (1/2)s1 + (1/2)e3 x4 = 1/2 + (1/2)x3 + (1/4)s1 + (1/4)e3 x1 − 2x2 + x3 − 3x4 − x1 + 3x2 + 2x3 + x4 x2 + x3 + 2x4 ≤ 1 = − 2 ≥ 1 + s1 = 1 = − 2 = 1 − a2 − e3 w = − a2 − a3 w = −3 + x1 − 4x2 − 3x3 − 3x4 − e3 + a3 w = − a2 − a3 z = − 7x3 − s1 + e3 z = 2x2 − 3x3

Degeneracy Basic solution is degenerate if some basic variable is zero x1=20 x2=60 x3=x4=x5=0 Basic solution is degenerate if some basic variable is zero multiple dictionaries 80 60 max 3x1 +2x2 x1 + x2 + x3 = 80 2x1 + x2 + + x4 = 100 6x1 + x2 + +x5 = 180 x1, x2, x3, x4, x5 ≥ 0 40 20 20 40 60 80 x1 = 20 + x3 − x4 x2 = 60 − 2x3 + x4 x5 = − 4x3 + 5x4 z = 180 − x3 − x4 x1 = 20 + 0.2x3 − 0.2x5 x2 = 60 − 1.2x3 + 0.2x5 x4 = 0.8x3 + 0.2x5 z = 180 − 1.8x3 − 0.2x5 x1 = 20 + 0.25x4 − 0.25x5 x2 = 60 − 1.5x4 + 0.5x5 x3 = 1.25x4 − 0.25x5 z = 180 − 2.25x4 + 0.25x5 optimal optimal ???

Upper bounded Simplex Anatomy of a dictionary Ratio test: 1. (different sign) x5: 3/1 = 3 2. (entering variable) x3: 2 3. (same sign) x6: (4−1)/2 = 1.5 (different sign) x5 = 3 − x1 + 2x2 − x3 + 3x4 x6 = 1 − x1 + 3x2 + 2x3 + x4 basis objective(s) z = − 2x2 + 3x3 (same sign) 0 ≤ x1, x2 , x4, x5 0 ≤ x3 ≤ 2 0 ≤ x6 ≤ 4 x3 enters bounds 0 ≤ x6’ ≤ 4 substitute x6 = 4 − x6’ x6 leaves (min ratio)

(all coefficients non-positive) Dual Simplex Anatomy of a dictionary Ratio test: x1: no constraint x2: 5/2 = 2.5 x3: no constraint x4: 7/4 = 1.75 (positive) negative value infeasible! x5 = − 3 − x1 + 2x2 − x3 + 4x4 x6 = 1 − x1 + 3x2 + 2x3 + x4 basis objective(s) z = − 5x2 − 3x3 − 7x4 dually feasible (all coefficients non-positive) x5 leaves x4 enters (min ratio)