1. IEOR 4004 Final Review part II.
April 30, 2014

2. Integer Programming Standard problems max cx Ax = b x ≥ 0 x integer
pure IP = all variables integer
mixed IP = some variables integer
Standard problems
Knapsack
Fixed-charge Problem (Facility location)
Cutting Stock Problem
Set cover
Travelling Salesman Problem
Job/Machine Scheduling Problems

3. IP formulations fixed charge (penalty) for a failing a constraint
alternative constraints (logical or)
if-then constraints
piece-wise linear objective
constraint
constraint
add
penalty
for failing
the constraint
f ≤ 0
f ≤ My
+py
y ∈ {0,1}
to the objective
f ≤ 0
f ≤ My
y ∈ {0,1}
g ≤ 0
g ≤ M(1-y)
f ≤ My
if f > 0 ,
then g ≤ 0
y ∈ {0,1}
g ≤ M(1-y)

4. optimal solution to the LP relaxation gives an upper bound
Branch-and-Bound
max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13
x1,x2,x3 ≥ 0 and integer
max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13
x1,x2,x3 ≥ 0
drop the
integrality
Linear Programming
(LP) relaxation
Integer Program
solve the LP relaxation
optimal solution to the LP relaxation gives an upper bound
branch on fractional variables (if exist)
how do we (re)-solve the subproblems quickly ?
Upper-bounded Dual Simplex
special subproblem (Knapsack, Transportation Problem)
fractional solution = solution to the LP relaxation

5. Upper-bounded Dual Simplex
Dually feasible solution
Optimal ?
1. check upper bounds
2. check lower bounds
1. substitute xi = B – xi’ where 0 ≤ xi ≤ B
2. pivot (dual ratio test) xi out, xj in the basis
given a dually feasible basic solution (all coeffs in z negative)
check if all basic variables satisfy their upper bounds
if some xi fails, if xi should be ≤ B but is > B, then substitute xi = B – xi’
check if all basic variables satisfy their lower bounds
if some xi < 0, then use dual ratio test to find xj such that the ratio b/a is smallest possible where:
a is the coefficient of xj in the equation for xi
(-b) is the coefficient of xj in the equation for z
a is positive
pivot xi out and xj into the basis
x1 = x2 – 4x3 + 3x4
x5 = x2 + 2x3 – 2x4
z = 5 – 3x2 – 5x3 – 4x4
x1 leaves, ratio test:
x2: 3 / 2 = 1.5
x4: 4 / 3 = 1.333
x4 enters

6. Branching tree optimal fractional solution max 6x1 + 5x2
s.t. 4x1 + 5x2 ≤ 16
8x1 + 5x2 ≤ 20
x1,x2 ≥ 0 & integer
optimal fractional solution
not an integer solution
branch
LP relaxation
x1 = 1
x2 = 2.4
z = 18
x2 ≤ = 2
x2 ≥ = 3
x2 ≤ 2
x2 ≥ 3
x1 = 1.25
x2 = 2
z = 17.5
x1 = 0.25
x2 = 3
z = 16.5
optimal
solution
x1 ≤ 1
x1 ≥ 2
x1 ≤ 0
x1 ≥ 1
x1 = 1
x2 = 2
z = 16
x1 = 2
x2 = 0.8
z = 16
x1 = 0
x2 = 3.2
z = 16
infeasible
candidate
solution
don’t need to branch (no solution here is better than z=16
and we already know a candidate solution of value 16)

7. not an integer solution
Example
Initial solution
x1 = 2.6
x2 = 0
x3 = 0
z = 31.2
max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 + x4 = 13
x1,x2,x3,x4 ≥ 0
max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13
x1,x2,x3 ≥ 0
x1 ≤ 2
x1 ≥ 3
x1 = 2
x2 = 1.5
x3 = 0
z = 30
infeasible
x1 = – 0.4x2 – 0.6x3 – 0.2x4
z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4
x2 ≤ 1
x2 ≥ 2
Subproblem: x1 ≥ 3
dictionary is not feasible anymore
x1 = 2
x2 = 1
x3 = 1/3
z = 29⅔
x1 = 1.8
x2 = 2
x3 = 0
z = 29.6
Repeat
How to quickly recover an optimum (without starting over) ?
Upper-bounded Dual Simplex method
x1 ≥ 3 substitute x1 = 3 + x5
x1 ≤ 2 substitute x1 = 2 – x1’
check upper bounds: OK
check lower bounds: -0.4 = x5 ≥ 0 x5 leaves, ratio test:
x2: no constraint
x3: no constraint
x4: no constraint
check upper bounds: OK
check lower bounds: -0.6 = x1’ ≥ 0 x1’ leaves, ratio test:
x2: 0.8/0.4 = 2
x3: 2.2/0.6 = 3⅔
x4: 2.4/0.2 = 12
x3 ≥ 1
optimal solution recovered
x3 = 0
x1 ≤ 1
x1 = 2
x1’ = x1 = 2–x1’ = 2
x2 = 1.5 x3 = 0 z = 30
x2 = x1’ – 1.5x3 – 0.5x4
z = 30 – 2x1’ – x3 – 2x4
x1’ = x x x4
z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4
x5 = – 0.4x2 – 0.6x3 – 0.2x4
z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4
x1 = 2
x2 = 1
x3 = 0
z = 28
x1 = 2
x2 = 0
x3 = 1
z = 29
x1 = 1
x2 = 4
x3 = 0
z = 28
infeasible
not an integer solution
branch on x2 ≤ 1 and x2 ≥ 2
infeasible LP
x2 enters
candidate solution
optimal
candidate solution

8. Knapsack 2.4 2 1⅔ max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13
value/
weight ⅔
Knapsack
Initial solution
max 12x1+ 4x2 + 5x3 s.t. 5x1+ 2x2 + 3x3 ≤ 13
x1,x2,x3 ≥ 0 and integer
x1 = 2.6
x2 = 0
x3 = 0
z = 31.2
optimal greedy strategy (for LP relaxation)
pick items by value/weight, highest first
take as many items of the same type as possible
x1 ≤ 2
x1 ≥ 3
x1 = 2
x2 = 1.5
x3 = 0
z = 30
infeasible
x2 ≥ 2 substitute x2 = 2 + x5
x1 ≥ 3 substitute x1 = 3 + x4
x2 ≤ 1
pick 2 units of item #1
then 1 unit of item #2
and then 1/3 of item #3
x2 ≤ 1
x2 ≥ 2
max 12x1+ 4x5 + 5x3 + 8 s.t. 5x1+ 2x5 + 3x3 ≤ 9
max 12x4+ 4x2 + 5x s.t. 5x4+ 2x2 + 3x3 ≤ -2
x3 ≥ 1 substitute x3 = 1 + x7
x1 = 2
x2 = 1
x3 = 1/3
z = 29⅔
x1 = 1.8
x2 = 2
x3 = 0
z = 29.6
max 12x1+ 4x2 + 5x7 + 5 s.t. 5x1+ 2x2 + 3x7 ≤ 10
x3 = 0
do not pick item #3
(1 unit unused in the bag)
pick 1.8 units of item #1
x1 ≤ 2
pick 2 units of item #1
and then 1.5 of item #2
(as much as possible)
pick 2 units of item #1
x1 = 2 substitute x1 = 2
x1 ≤ 1
pick 1 units of item #1
then 2 of item #5
(i.e. 4 units of item #2)
x3 = 0
x3 ≥ 1
x1 ≤ 1
x1 = 2
max 4x5 + 5x s.t. 2x5 + 3x3 ≤ -1
x1 = 2
x2 = 1
x3 = 0
z = 28
x1 = 2
x2 = 0
x3 = 1
z = 29
x1 = 1
x2 = 4
x3 = 0
z = 28
infeasible
candidate solution
optimal
candidate solution

9. Cutting Planes only for pure IPs solve LP relaxation
optimal
fractional
solution
max 6x1 + 5x2
s.t. 4x1 + 5x2 ≤ 16
8x1 + 5x2 ≤ 20
x1,x2 ≥ 0 & integer 1 2 3 4 5
x1 + x2 ≤ 3
x1,x2 ≥ 0 & integer
new
fractional
optimum
only for pure IPs
solve LP relaxation
if not an integer optimum, make the feasible region smaller by cutting out fractional points (without removing any integer points)
how to generate cuts ?
Gomory cuts

10. Gomory cuts max 6x1 + 5x2 s.t. 4x1 + 5x2 ≤ 16 8x1 + 5x2 ≤ 20
optimal
solution
max 6x1 + 5x2
s.t. 4x1 + 5x2 ≤ 16
8x1 + 5x2 ≤ 20
x1 + x2 ≤ 3
x1,x2 ≥ 0 & integer
max 6x1 + 5x2
s.t. 4x1 + 5x2 + x3 = 16
8x1 + 5x2 + x4 = 20
x1 + x2 +x5 = 3
x1,x2,x3,x4,x5 ≥ 0 & integer 1 2 3 4 5
optimal
fractional
solution
x1 = – x x5
x2 = x6 – 3x5
x3 = – x x5
x4 = x6 – x5
z = – x6 – 3x5
x1 = 1⅔ – ⅓x4 + 1⅔x5
x2 = 1⅓ + ⅓x4 – 2⅔x5
x3 = 2⅔ – ⅓x4 + 6⅔x5
z = 16⅔ – ⅓x4 – 3⅓x5
split to whole and fractional part
x6 = – ⅔ + ⅓x4 + ⅓x5
z = 16⅔ – ⅓x4 – 3⅓x5
x1 = 1⅔ – ⅓x ⅔x5
1+⅔
0+⅓
2–⅓
negative
coeffients
x1 – 2x5 – = ⅔ – ⅓x4 – ⅓x5
≤ 0
< 1
Use Dual Simplex to
recover the optimum
cut
whole part
fractional part
positive
constant

11. Goal: – value of the formula for a target state (in the last stage)
– save on calculation by storing the intermediate results fi(j)
Dynamic Programming
optimal substructure property
time
(months, years)
budget
(available resources)
Break down the problem into stages
each stage has possible states
next stage depends only on the previous stage
identify possible decisions/transitions at each stage
find a formula to evaluate (by recursion)
Evaluate stage by stage
fi ( j ) = optimum value
for the first i stages that
ends in state j of stage i 1 1 1 2 2 2
e.g. cost of cheapest production plan for
first i months where we end up with
j items in inventory at the end of month i 3 3 3
(inventory in month i+1 only depends on
production and inventory level in month i)
stage 1
stage 2
stage 3

12. Dynamic Programming #1 #2 #36 1 2 3 4 5 #1 #2 #3
cash 1 2 3 4 5 6
start #1 9 16 23 30 37 44 #2 10 19 26 33 40 47 #3 49
#1. x1 > 0 thousand $ yields c1(x1)=7x1+2 thousand $
Investments:
#2. x2 > 0 thousand $ yields c2(x2)=3x2+7 thousand $
#3. x3 > 0 thousand $ yields c3(x3)=4x3+5 thousand $
We can invest up to $6,000.
c1(0)=c2(0)=c3(0)=0
green arrow = best choice of k
decision in stage i = how much we invest into #i
(all we need to know is how much money we have available for investing)
fi( j ) = maximum profit we can get by investing
j thousand dollars into first i investments
f0( j ) = 0
profit from #i
leftover cash
fi( j ) =
max { }
ci(k)
+ fi-1( )
j – k
k ∈ {0,1,...,j}
best possible profit
with the leftover cash
how much
we invest in #i

13. Traveling Salesman Problem
Find a shortest route that goes through all cities
and comes back where it starts
Map of cities
Shortest route

14. Traveling Salesman Problem
Work backwards – which is the
last vertex visited before v ? s v
f(S,v) = min f(S\{x}, x) + cxv x
x ∈ S
(x,v) ∈ E
initial condition:
f(Ø,s) = 0
S\{x} S
solution:
f(V\{s},s)
f(S,v) = length of
optimal
substructure
property
a shortest route starting at s
ending at v and visiting all nodes in S
brute force (trying all routes)
is O(n!) time and O(1) space
complexity: O(n22n) time and space

15. Good luck on the final exam.