1. MS&E 348 The L-Shaped Method: Theory and Example 2/5/04 Lecture

2. Review: Dantzig-Wolfe (“DW”) Decomposition We recognize delayed column generation as the centerpiece of the decomposition algorithm Even though the master problem can have a huge number of columns, a column is generated only after it is found to have a negative reduced cost and is about to enter the basis The subproblems are smaller LP problems that are employed as an economical search method for discovering columns with negative reduced costs A variant can be used whereby all columns that have been generated in the past are retained

3. Review: Benders Decomposition Benders decomposition uses delayed constraint generation and the cutting plane method, and should be contrasted with DW that uses column generation Benders is essentially the same as DW applied to the dual Similarly as for DW, we have the option of discarding all or some of the constraints in the relaxed primal that have become inactive

4. Review: Two-Stage Problem with Fixed Recourse min c T x+ ∑ k p k q k T y k s.t. Ax=b T k x+Wy k =h k k=1,…,K scenarios x ≥ 0 y k ≥ 0 In extensive form (“EF”) for a discrete finite set of scenarios Technology & Recourse matrices min z = c T x+ E w [min q(w) T y(w)] s.t. Ax=b T(w)x+Wy(w)=h(w) x ≥ 0 y(w) ≥ 0 As defined by Dantzig (1955) and Beale (1955) Technology & Recourse matrices Scenario

5. Why the L-Shaped Method? … … A T1T1 T2T2 TKTK W W W Block structure of the 2-stage extensive form ATAT T1T1 T2T2 TKTK WTWT WTWT WTWT … … Block structure of the 2-stage dual Hence the name: L-shaped method!

6. Connection between Benders/DW and the L-Shaped Method Given this block structure, it seems natural to exploit the dual structure by performing a DW (1960) decomposition (inner linearization) of the dual … or a Benders (1962) decomposition (outer linearization) of the primal The method has been extended in stochastic programming to take care of feasibility questions and is known as Van Slyke and Wets’ (1969) L- shaped method It is a cutting plane technique

7. Comments on the L-Shaped Method - The method consists of solving an approximation of the recourse function by using an outer linearization of Q. Two types of constraints are sequentially added: (i) feasibility cuts determining { x | Q(x) < +∞} and (ii) optimality cuts that are linear approximations to Q on its domain of finiteness - Observe that solving P1 is equivalent to solving P2 min c T x + Q(x) s.t. x Є K 1 ∩ K 2 K 1 = { x | Ax=b, x≥0 } K 2 = { x | Q(x) < ∞ } P1: min c T x + Ө s.t. x Є K 1 ∩ K 2 Q(x) ≤ Ө P2:

8. The L-Shaped Algorithm Step 0. Set r = s = v = 0 (these are indices initially set at zero) Step 1. Set v = v+1 (v is the iteration index) and solve the LP (1) - (3) min z = c T x + Ө s.t. Ax=b D l x ≥ d l l=1,…,r feasibility cuts x ≥ 0 and Ө is a free real number G s x + Ө ≥ g s l=1,…,s optimality cuts (2) (3) (1) Let (x v, Ө v ) be an optimal solution. Detail for Initialization: if no constraint (3) is present, Ө v is set equal to -∞ and is not considered in the computation of x v.

9. The L-Shaped Algorithm (Cont’d) Step 2. For k = 1, …, K solve the LP (Phase I Simplex) min w k ’ = e T u + + e T u - s.t. W y + I u + - I u - = h k - T k x v (5) (4) y ≥ 0, u + ≥ 0, u - ≥ 0 where e T = (1, …,1) until for some k, the optimal value w k ’ > 0 In this case, let σ v be the associated simplex multipliers and define d r+1 = (σ v ) T h k D r+1 = (σ v ) T T k and to generate a feasibility cut of type (2). Set r = r + 1, add to the constraint set (2), and return to Step 1. If for all k, w k ’ = 0, go to step 3 (7) (6) Identity matrix ∆!: these are obviously ≠

10. The L-Shaped Algorithm (Cont’d) Step 3. For k = 1, …, K solve the LP min w k = q k T y s.t. W y = h k - T k x v (8) y ≥ 0 Let Π k v be the simplex multipliers associated with the optimal solution of problem k of type (8). Define and G s+1 = ∑ k p k (Π k v ) T T k (9) (10) g s+1 = ∑ k p k (Π k v ) T h k Let w v = g s+1 – G s+1 x v If Ө v ≥ w v, stop as x v is an optimal solution. Otherwise, set s = s + 1, add to the constraint set (3) and return to Step 1.

11. Example min z = 0 + Q(x,ξ k ) (we assume c T = 0) where Q(x,ξ) = We also assume 0 ≤ x ≤ 10 andξ = Solve this example : ξ - x if x ≤ ξ x - ξ if x ≥ ξ 1 w.p. 1/3 2 w.p. 1/3 4 w.p. 1/3 (i) directly (ii) by using the L-shaped algorithm

12. Direct Solution ξ 1 =1ξ 2 =2ξ 3 =4 124 x Q(x,ξ k ) w.p. 1/3 1/3 1/3 7/3 5/3 4/3 1 Q(x) = ∑p k Q(x,ξk) can be constructed directly… x |Slope| = 1

13. Using the L-Shaped Algorithm - There are no feasibility cuts as K 1 = R (there is no Ax=b constraint) and K 2 = R (Q(x) is defined everywhere) -We can reformulate this problem to fit the notation previously used Each subproblem can be formulated as: min w k = q k T y s.t. W y = h k - T k x v y ≥ 0 min y1 s.t. y1 – y2 = ξ - x y1 – y3 = - ξ + x 1 -1 0 1 0 -1 W = ξ-ξ ξ-ξ hk=hk= 1 Tk=Tk= with,, which is equivalent to…

14. Iteration by Iteration Iteration 1 We solve min Ө We add the cut of type (3): Ө ≥ 7/3 - x s.t. 0 ≤ x ≤ 10 Since at initialization, we assume Ө 1 = -∞ Since x is undetermined, we assume we start at x 1 =0 There’s no feasibility issue, we go to Step 3 of the algorithm We find G 1 = ∑ k ⅓ [1 0] = 1 1 g 1 = ∑ k ⅓ [1 0] = 7/3 ξk-ξk ξk-ξk w 1 = g 1 – G 1 x 1 = 1 and w 1 > Ө 1 and (x 1 = 0, Ө 1 = -∞) is not optimal Simplex multipliers are the same for all three scenarios…

15. Graphically… 124 7/3 5/3 4/3 1 Q(x) x First Optimality Cut

16. Iteration by Iteration (Cont’d) Iteration 2 We solve min Ө We add the optimality cut: Ө ≥ x – 7/3 s.t. Ө ≥ 7/3 – x 0 ≤ x ≤ 10 We find (x 2 = 10, Ө 2 = -23/3) There’s no feasibility issue, we go to Step 3 of the algorithm We find G 2 = ∑ k ⅓ [0 1] = -1 1 g 2 = ∑ k ⅓ [0 1] = -7/3 ξk-ξk ξk-ξk w 2 = g 2 – G 2 x 2 = 23/3 and w 2 > Ө 2 and (x 2 = 10, Ө 2 = -23/3) is not optimal Simplex multipliers are the same for all three scenarios…

17. Graphically… 124 7/3 5/3 4/3 1 Q(x) x First Optimality Cut Second Optimality Cut

18. Iteration by Iteration (Cont’d) Iteration 3 We solve min Ө We add the optimality cut: Ө ≥ (x+1)/3 s.t. Ө ≥ 7/3 – x Ө ≥ x - 7/3 0 ≤ x ≤ 10 We find (x 3 = 7/3, Ө 3 = 0) We find G 3 = ⅓ [0 1] + ⅓ [0 1] + ⅓ [1 0] = -1/3 1 g 3 = 1/3 w 3 = g 3 – G 3 x 3 = 10/9 and w 3 > Ө 3 and (x 3 = 7/3, Ө 3 = 0) is not optimal Simplex multipliers are NOT the same for all three scenarios… 1 1

19. Iteration by Iteration (Cont’d) Iteration 4 We add the optimality cut: Ө ≥ (5-x)/3 We find (x 4 = 3/2, Ө 4 = 5/6) which is not optimal Iteration 5 We find (x 5 = 2, Ө 5 = 1) which is OPTIMAL

20. Graphically… 124 7/3 5/3 4/3 1 Q(x) x OC 1 Optimality Cut (“OC”) OC 2 OC 4 OC 3 (x*=2, Ө 5 =1) OPTIMAL