1. IEOR 4004 Midterm Review (part I)
March 10, 2014

2. Summary Modeling optimization problems Linear program Simplex method
Algorithm
Initialization
Variants
Sensitivity analysis (in part II)
Duality (in part II)
Upper bounds and Shadow prices
Complementary slackness

3. Mathematical modeling
Simplified (idealized) formulation
Limitations
Only as good as our assumptions/input data
Cannot make predictions beyond the assumptions
We need more maps

4. Mathematical modeling
Problem
simplification
Model
formulation
Model
Algorithm
selection
Numeric
calculation
Problem
Solution
Interpretation
Sensitivity analysis

5. Mathematical modeling
Deterministic = values known with certainty
Stochastic = involves chance, uncertainty
Linear, non-linear, convex, semi-definite

6. Formulating an optimization problem
Linear program
Linear
objective
Decision variables
Objective
Constraints
Domains
x1, x2, x3, x4 + x1
(1 − 2x2)2
Minimize
2x2 + 3x3
(x1)2 + (x2)2 + (x3)2 + (x4)2 ≤ 2
x1 − 2x2 + x3 − 3x4 ≤ 1
− x1 + 3x2 + 2x3 + x4 ≥ − 2
− 2x1x3 = 1
Linear
constraints
x1 ≥ 0
x3 in {0,1}
x4 in [0,1]
x2 ≠ 0.5
Sign restriction

7. Linear program
solution = values of variables that together satisfy all constraints
Feasible solution = satisfies also sign constraints
Infeasible solution = not feasible
Basis = set of m variables (where m = # of equations)
Variable in basis = basic variable, all other non-basic
Basic solution = set all non-basic variables to zero
{x1, x2}
x1 − 2x2 + x3 − 3x4 = 1
− x1 + 3x2 + 2x3 + x4 = − 2
x1 = − 1
x2 = −1
x3 = 0
x4 = 0
x1 = 4/3
x2 = 0
x3 = −1/3
x4 = 0
x1 ≥ 0

8. Linear program Feasible region = set of all feasible solutions
LP is Infeasible if feasible region empty
LP is Unbounded if the objective function is unbounded over the feasible region
objective
function
grows
beyond
bounds
objective
function
growth
is limited

9. Simplex algorithm Converting to standard form Add slack variables
Substitute negative variables
Substitute unrestricted variables
Dictionary
x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4−
x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4−
− 3x4+ + 3x4−
+ x4+ − x4−
x1 − 2x2
− x1 + 3x2
− x3’
− 2x3’
+ x3
+ 2x3
+ x5 =
− x6 =
− 3x4
+ x4 ≤ ≥ 3
− 2
z = − 2x2 + 3x3’
x1, x2 ≥ 0
x5, x6 ≥ 0
x3 ≤ 0
x3’ ≥ 0
x4+, x4− ≥ 0

10. Simplex method (maximization)
START
x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4−
x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4−
Initial feasible solution
z = − 2x2 + 3x3’ xj
Pivot xj to the basis
Optimality test
Are all coefficients
in z non-positive ?
Ratio test
Find min b/a
a = coeff of xi in xj
b = value of xj
a, b opposite signs
Variable selection xi NO
YES
END
Alternative
solutions?
optimal solution found
xj does not exist
LP is unbounded

11. Simplex algorithm Anatomy of a dictionary Ratio test: x5: 3/1 = 3
x6: no constraint
(different sign)
(same sign)
x5 = 3 − x1 + 2x2 − x3’ + 3x4+ − 3x4−
x6 = 2 − x1 + 3x2 + 2x3’ + x4+ − x4−
basis
objective(s)
z = − 2x2 + 3x3’
zero
coeff
negative
coeff
positive
coeff
x3’ enters
x5 leaves
(min ratio)

12. Simplex/Graphical method
x3, x4 are slack
variables
Illustration
profit:
180
150 80
max 3x1 +2x2
x1 + x2 + x3 = 80
2x1 + x2 + x4 = 100
x1, x2, x3, x4 ≥ 0
max 3x1 +2x2
x1 + x2 ≤ 80
2x1 + x2 ≤ 100
x1, x2 ≥ 0 z
start: x1=0, x2=0, x3=80, x4=100
x1 increases until x4=0 60
x2 increases until x3=0
optimal 40 x3 x1 x4
isoprofit
line 20 x2
3x1 +2x2 = profit 20 40 60 80

13. Initialization - Phase I
Goal: make all original variables non-basic (=0)
If all-0 doesn’t satisfy a constraint, add artificial variable ai:
add +ai if the right-hand side is > 0
add –ai if the right-hand side < 0
Add slack variables (where needed)
New objective: minimize the sum of artificial variables
Starting basis: all artificial + slack variables
from equations without artificial variables
If optimal value negative, then
LP Infeasible
Else drop artificial variables and introduce z
Maximize w = − a2 − a3
z = 2x2 − 3x3
s1 = 1 − x1 + 2x2 − x3 + 3x4
a2 = 2 − x1 + 3x2 + 2x3 + x4
a3 = 1 − x2 − x3 − 2x4 + e3
x1 = 5/2 − (7/2)x3 + (1/4)a2 − (7/4)a3 − (5/4)s1 + (7/4)e3
x2 = − x3 + (1/2)a2 − (1/2)a3 − (1/2)s1 + (1/2)e3
x4 = 1/2 + (1/2)x3 − (1/4) a2 − (1/4)a3 + (1/4)s1 + (1/4)e3
x1 = 5/2 − (7/2)x3 − (5/4)s1 + (7/4)e3
x2 = − x3 − (1/2)s1 + (1/2)e3
x4 = 1/2 + (1/2)x3 + (1/4)s1 + (1/4)e3
x1 − 2x2 + x3 − 3x4
− x1 + 3x2 + 2x3 + x4
x2 + x3 + 2x4
≤ 1
= − 2
≥ 1
+ s1
= 1
= − 2
= 1
− a2
− e3
w = − a2 − a3
w = −3 + x1 − 4x2 − 3x3 − 3x4 − e3
+ a3
w = − a2 − a3
z = − x3 − s e3
z = 2x2 − 3x3

14. Degeneracy Basic solution is degenerate if some basic variable is zero
x1=20
x2=60
x3=x4=x5=0
Basic solution is degenerate if some basic variable is zero
multiple dictionaries 80 60
max 3x1 +2x2
x1 + x2 + x3 = 80
2x1 + x x4 = 100
6x1 + x x5 = 180
x1, x2, x3, x4, x5 ≥ 0 40 20 20 40 60 80
x1 = 20 + x3 − x4
x2 = 60 − 2x3 + x4
x5 = − 4x3 + 5x4
z = 180 − x3 − x4
x1 = x3 − 0.2x5
x2 = 60 − 1.2x x5
x4 = 0.8x x5
z = 180 − 1.8x3 − 0.2x5
x1 = x4 − 0.25x5
x2 = 60 − 1.5x x5
x3 = 1.25x4 − 0.25x5
z = 180 − 2.25x x5
optimal
optimal ???

15. Upper bounded Simplex Anatomy of a dictionary Ratio test:
1. (different sign)
x5: 3/1 = 3
2. (entering variable)
x3: 2
3. (same sign)
x6: (4−1)/2 = 1.5
(different sign)
x5 = 3 − x1 + 2x2 − x3 + 3x4
x6 = 1 − x1 + 3x2 + 2x x4
basis
objective(s)
z = − 2x2 + 3x3
(same sign)
0 ≤ x1, x2 , x4, x5
0 ≤ x3 ≤ 2
0 ≤ x6 ≤ 4
x3 enters
bounds
0 ≤ x6’ ≤ 4
substitute
x6 = 4 − x6’
x6 leaves
(min ratio)

16. (all coefficients non-positive)
Dual Simplex
Anatomy of a dictionary
Ratio test:
x1: no constraint
x2: 5/2 = 2.5
x3: no constraint
x4: 7/4 = 1.75
(positive)
negative
value
infeasible!
x5 = − 3 − x1 + 2x2 − x3 + 4x4
x6 = 1 − x1 + 3x2 + 2x x4
basis
objective(s)
z = − 5x2 − 3x3 − 7x4
dually feasible
(all coefficients non-positive)
x5 leaves
x4 enters
(min ratio)