1. Chapter 10 Advanced Topics in Linear Programming
to accompany
Operations Research: Applications and Algorithms
4th edition
by Wayne L. Winston
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

2. 10.1 The Revised Simplex Algorithm
Section 6.2 demonstrated how to create an optimal tableau from an initial tableau, given an optimal set of the basic variables.
The same results can be used to create a tableau corresponding to any set of basic variables.
To create a tableau for any set of basic variables BV, view the notation shown on the following slide.
Assume the LP has m constraints.

3. BV = any set of basic variables (the first element of BV is the basic variable in the first constraint, the second basic variable is the basic variable in the second constraint, etc. BVj is the basic variable for the constraint j in the desired tableau.
b = right-hand side vector of the original tableau’s constraints
aj = column for xj in the constraints of the original problem
B = m x m matrix whose jth column is the column for BVj in the original constraints
cj = coefficients of xj in the objective function
cBV = 1 x m row vector whose jth element is the objective function coefficient for BVj
ui = m x 1 column vector with ith element 1 and all other elements equal to zero.

4. Summarizing the formulas of Section 6.2:
B-1aj = column for xj in the BV tableau
cBVB-1aj – cj = coefficient of xj in row 0
B-1b = right-hand side of constraints in BV tableau
cBVB-1ui = coefficient of slack variable si, in BV row 0
cBVB-1(-ui) = coefficient of excess variable ei, in BV row 0
M + cBVB-1ui = coefficient of artificial variable ai in BV row 0
cBVB-1b = right-hand side of BV row 0

5. If we know BV, B-1, and the original tableau, the formulas on the previous slide enable us to compute any part of the simplex tableau for any set of basic variables BV.
This means that if a computer is programmed to perform the simplex algorithm, all the computer needs to store on any pivot is the correct set of basic variables, B-1, and the initial tableau. This idea is the basis for the revised simplex algorithm.

6. A summary of the revised simplex method (max problem) is:
Step 0 Note the columns from which the current B-1 will be read. Initially, B-1 = 1.
Step 1 For the current tableau, compute cBVB-1
Step 2 Price out all nonbasic variables in the current tableau. If each nonbasic variable prices out nonnegative, the current basis is optimal. If the current basic is not optimal, enter into the basis the nonbasic variable with the most negative coefficient in row 0. Call this variable xk.

7. Step 3 To determine which row in which xk enters the basis, compute xk’s column in the current tableau (B-1ak) and compute the rhs of the current tableau (B-1b). Then use the ratio test to determine the row in which xk should enter the basis. We now know the set of basic variables (BV) for the new tableau.
Step 4 Use the column for xk in the current tableau to determine the ERO’s needed to enter xk into the basis. Perform these ERO’s on the current B-1. This will yield the new B-1. Return to Step 1.

8. Most linear programming codes use some version of the revised simplex method to solve LPs.
Since knowing the current tableau’s B-1 and the initial tableau is all that is needed to obtain the next tableau, the computational effort required to solve an LP depends primarily upon the size of B-1.
In some cases involving models with a large number of constraints (equations), solving the dual in lieu of the primal will be much easier computationally.

9. 10.2 The Product Form of the Inverse
Much of the computation in the revised simplex algorithm is concerned with updating B-1 from one tableau to the next.
A matrix (like E) that differs from the identity matrix in only one column is called an elementary matrix.
It can shown that: B-1 for the new tableau = E(B-1) for the current tableau

10. This equation is called the product form of the inverse
This equation is called the product form of the inverse. Most linear programming computer codes utilize the revised simple method and compute successive B-1’s by using the product form of the inverse.

11. 10.3 Using Column Generation to Solve Large-Scale LPs
For LPs that have many variables, column generation can be used to increase the efficiency of the revised simplex algorithm.
Column generation is also a very important component of the Dantzig-Wolfe decomposition algorithm.

12. Example 2: Odds and Evens
Woodco sells 3-ft, 5-ft, and 9-ft pieces of lumber.
Woodco’s customers demand 25 3-ft boards, 20 5-ft boards and 15 9-ft boards.
Woodco, who must meet its demands by cutting up 17-ft boards, wants to minimize the waste incurred.
Formulate an LP to help Woodco accomplish its goal, and solve the LP by column generation.

13. Example 2: Solution
Each decision corresponds to a way in which a 17-ft board can be cut.
Shown are the sensible ways to cut a 17-ft board
Number of
Combination
3-ft boards
5-ft boards
9-ft boards
Waste(feet) 1 5 2 4 3 6

14. Ex. 2 – Solution continued
Woodco’s objective function is min z = x1 + x2 + x3 + x4 + x5 + x6
Constraints
At least 25 3-ft boards must be cut. (5x1 + 4x2 + 2x3 + 2x4 + x5 ≥ 25
At least 20 5-ft boards must be cut. (x2 + 2x3 + x5 + 3x6 ≥ 20
At least 15 9-ft boards must be cut. (x4 + x5 ≥ 15)
xi should be required to assume integer values
After creating the LP, each nonbasic variable should be priced out to determine which variable should enter the basis.

15. Ex. 2 – Solution continued
However, in a large scale cutting stock problem there may be thousands of variables, so this is where column generation comes into play.
The idea of column generation is to search efficiently for a column that will price out favorably.
Find the combination that prices out most favorably by solving the following knapsack problem
max z = 1/5a3 + 1/3a5 + a9-1
s.t. 3a3 + 5a5+ 9a9 ≤ 17
a3, a5, a9 ≥ 0; a3, a5, a9 integer

16. Ex. 2 – Solution continued
Because it is a knapsack problem (without 0-1 restrictions on the variables), it can easily be solved by using the branch-and-bound procedure.
Branch-and-bound trees are the result.

17. 10.4 – The Dantzig-Wolfe Decomposition Algorithm
In many LPs, the constraints and variables may be decomposed in the following manner:
Constraints in Set k+1 involve any variable. The constraints in Set k+1 are referred to as the central constraints.
LPs that can be decomposed in this fashion can be solved efficiently by the Dantzig-Wolfe decomposition algorithm.
Constraints in Set 1 only involve variables in Variable Set 1
Constraints in Set 2 only involve variables in Variable Set 2
….. ….
Constraints in Set k only involve variables in Variable Set k.

18. Example 3: Decomposition
SteelCo manufactures two type of steel (steel 1 and steel 2) at two locations (plant 1 and plant 2) requiring the following resources according as shown.
Product (1 ton)
Iron Required (tons)
Coal Required (tons)
Blast Furnace Time Required (hours)
Steel 1 at Plant 1 8 3 2
Steel 2 at Plant 1 6 1
Steel 1 at Plant 2 7
Steel 2 at Plant 2 5

19. Ex. 3 - continued Each day the following resources are available:
Steel selling prices:
Steel shipping costs:

20. Ex. 3 - continued
Each plant has its own coal mine and coal cannot be shipped between plants. Each plant has a different type of furnace requiring different resources. Iron ore is mined from a single mine located midway between the plants.
Formulate and solve an LP to maximize SteelCo’s Revenue.

21. Example 3: Solution Define:
SteelCo’s revenue is given by: 170(x1 + x3) + 160(x2 + x4)
SteelCo’s shipping costs are given by: (x1 + x2) + 100(x3 + x4)
x1 = tons of steel 1 produced daily at plant 1
x2 = tons of steel 2 produced daily at plant 1
x3 = tons of steel 1 produced daily at plant 2
x4 = tons of steel 2 produced daily at plant 2

22. Ex. 3 – Solution continued
Therefore, SteelCo wants to maximize: z = ( )x1 + ( )x (170 – 100)x3 + ( )x4 or: z = 90x1 + 80x2 + 70x3 + 60x4
SteeCo’s LP is
max z = 90x1 + 80x2 + 70x3 + 60x4
s.t. 3x x ≤ (plant 1 coal constraint)
2x x ≤ 10 (plant 1 furnace constraint)
3x3 + 2x4 ≤ (plant 2 coal constraint)
x3 + x4 ≤ (plant 1 furnace constraint)
8x x x3 + 5x4 ≤ (iron ore constraint)
x1, x2, x3, x4 ≥ 0

23. Ex. 3 – Solution continued
Using the definition of decomposition, the SteelCo LP may be decomposed into
Variable Set 1 x1 and x2 (plant 1 variables)
Variable Set 2 x3 and x4 (plant 2 variables)
Constraint 1 Plant 1 coal and furnace constraints
Constraint 2 Plant 2 coal and furnace constraints
Constraint 3 Iron ore constraint

24. Ex. 3 – Solution continued
Constraint Set 1 and Variable Set 1 involve activities at Plant 1 and do not involve x3 and x4 (which represent plant 2 activities).
Constraint Set 2 and Variable Set 2 involve activities at Plant 2 and do not involve x1 and x2 (which represent plant 1 activities).
Constraint 3 may be thought of as a centralized constraint that interrelates the two sets of variables

25. Assuming each sub problem in the SteelCo LP has a bounded feasible region, the results depend upon the following theorem.
Suppose the feasible region for an LP is bounded and the extreme points (or basic feasible solutions) of the LP’s feasible region are P1, P2, …, Pk. Then any point x in the LP’s feasible region may be written as a linear combination of P1, P2, …, Pk. Or,
x = μ1P1 + μ2P2 + … + μkPk
Moreover, the weights of μ1, μ2 , …, μk may be chosen such that:
μ1+ μ2 + … + μk = 1 and μi ≥ 0 for i = 1, 2, …, k

26. Any linear combination of vectors which satisfy. μ1+ μ2 + … + μk = 1
Any linear combination of vectors which satisfy μ1+ μ2 + … + μk = 1 and μi ≥ 0 for i = 1, 2, …, k is called a convex combination.
To explain the basic ideas of the Dantzig-Wolfe deposition algorithm, assume the set of variables has been decomposed into Set 1 and Set 2 as in the SteelCo LP.
The steps shown can be generalized where decomposition results in more than just two sets of variables.

27. Step 1 Let the variables in Variable Set 1 by x1, x2, …, xn1
Step 1 Let the variables in Variable Set 1 by x1, x2, …, xn1. Express the variables as a convex combination of the extreme points of the feasible region for Constraint Set 1. If we let P1, P2, …, Pk be the extreme points of the feasible region, then any point in the feasible region can be written in the form:
= μ1P1 + μ2P2 + … + μkPk
where: μ1+ μ2 + … + μk = 1 and μi ≥ 0 for i = 1, 2, …, k

28. Step 2 Let the variables in Variable Set 2 by xn1+1, xn1+2, …, xn
Step 2 Let the variables in Variable Set 2 by xn1+1, xn1+2, …, xn. Express the variables as a convex combination of the extreme points of the feasible region for Constraint Set 2. If we let Q1, Q2, …, Qm be the extreme points of the feasible region, then any point in Constraint Set 2’s feasible region can be written in the form:
= λ1Q1 + λ2Q2 + … + λmQm
where: λ1+ λ2 + … + λm = 1 and λi ≥ 0 for i = 1, 2, …, m

29. Step 3 Using the equations on slides 33 and 34, express the LP’s objective function and centralized constraint in terms of μi‘s and λi‘s. After adding the constraints (called convexity constraints) μ1+ μ2 + … + μk = λ1+ λ2 + … + λm = 1 and the sign restrictions, we obtain the following LP which is referred to as a restrictive master shown on the next slide.

30. Max (or min)
[objective function in terms of μi‘s and λi‘s]
s.t. [central constraints in terms of μi‘s and λi‘s]
μ1+ μ2 + … + μk = 1
λ1+ λ2 + … + λm = 1
μi ≥ 0 for i = 1, 2, …, k
λi ≥ 0 for i = 1, 2, …, m
Step 4 Assume that a basic feasible solution for the restricted master is readily available. Then use the column generation method to solve the restricted master.

31. = μ1P1 + μ2P2 + … + μkPk = λ1Q1 + λ2Q2 + … + λmQm
Step 5 Substitute the optimal values of the μi‘s and λi‘s found in Step 4 into the equations. This will yield the optimal values of x1, x2, … , xn
= μ1P1 + μ2P2 + … + μkPk
= λ1Q1 + λ2Q2 + … + λmQm

32. 10.5 The Simplex Method for Upper-Bounded Variables
Often LPs contain many constraints of the form xi ≤ ui, that provide an upper bound on xi which are called an upper-bound constraint.
To efficiently solve an LP with upper-bound constraints, allow the variable xi to be nonbasic if xi =0 or if xi = ui.
Whenever xi is needed to equal its upper bound of ui, simply replace xi by ui-xi’. This is called an upper-ound substitution.

33. Three possible occurrences, or bottlenecks, can restrict the amount by which xi can be increased.
xi cannot exceed its upper bound of ui
xi increases to a point where it causes one of the current basic variables to become negative. The smallest of xi that will cause one of the current basic variables to become negative may be found by expressing each basic variable in terms of xi.
xi increases to a point where it causes one of the current basic variables to exceed its upper bound. As in bottleneck 2, the smallest value of xi for which this bottleneck occurs can be found by expressing each basic variable in terms of xi.

34. 10.6 Karmarkar’s Method for Solving LPs
Karmarkar’s method for solving LPs is a polynomial time algorithm.
Karmarkar’s method is applied to an LP in the following form.
The following three concepts play a key role in Karmarkar’s method
Projection of a vector onto the set of x satisfying Ax=0
Karmarkar’s centering transformation
Karmarkar’s potential function
min z =cx
s.t Ax = 0
x1+ x2 + ∙∙∙ + xn = 1
x ≥ 0

35. The n-dimensional unit simplex S is the set of points [x1+ x2 + ∙∙∙ + xn]T satisfying x1+ x2 + ∙∙∙ + xn =1 and xj ≥ 0, j = 1, 2, ∙∙∙ n.
Karmarkar’s centering transformation has the following properties.