1. Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity » Polynomial time algorithms » Intractability » NP-complete problems 2.Basic Properties of Linear Programming » Forms of LP » Basic feasible solutions » Geometry of LP 3.The Revised Simplex Method

2. Today 1.The R evised Simplex Algorithm » Basics » The algorithm » Getting an Initial Feasible Solution 2.Computational Aspects » Updating the basis » Convergence and Degeneracy » The Revised Simplex Method 3.Duality » Dual LP » The Duality Theorem » Complementary Slackness » The Dual Simplex Algorithm 2008/3/42Shi-Chung Chang, NTUEE, GIIE, GICE

3. Reading Assignments: 1.Sections 3.1-3.3, 4.1-4.2 of [PaS 8 3] 2.Sections 4.1-4.5 of [Lue 8 4] Next Time: sections 4.3,4.4 and Chapter 5 of [PaS 83] 2008/3/43Shi-Chung Chang, NTUEE, GIIE, GICE

4. §2.2 The Revised Simplex Method The simplex method for LP : G.B. Dantzig 1951 Key idea: Phase1: find a bfs of Note that an LP may not have a solution Example min s.t. 2008/3/44Shi-Chung Chang, NTUEE, GIIE, GICE

5. Phase 2 Allow one of the zero components of the bfs to become positive and force one of the original positive components to become zero. => How to pick “entering” and “leaving” component Cost Traditional form of the simplex method: Tableau => read by yourself Here we consider matrix form for conciseness of presentation and later developments. 2008/3/45Shi-Chung Chang, NTUEE, GIIE, GICE

6. SLP: s.t. Assume P(A)=m. Partition A=[B:D] B: m linearly indep. columns of A (Assume the first m cols.) (SLP) subject to (3.1) 2008/3/46Shi-Chung Chang, NTUEE, GIIE, GICE

7. If and a basic solution When from (3.1) (note that may not ≥0 ) (3.2) Substituting(3.2) into the cost function (3.3) Define (3.4) as the relative cost vector => To minimize Z, we need only adjust Q: How? 2008/3/47Shi-Chung Chang, NTUEE, GIIE, GICE

8. Example subject to =2 + =5 + =6 In matrix form. 2008/3/48Shi-Chung Chang, NTUEE, GIIE, GICE

9.  2008/3/49Shi-Chung Chang, NTUEE, GIIE, GICE

10. Define the vector of simplex multipliers Compute in two steps: Assume that is non degenerate If is optimal and it does not pay to increase If then the cost Z can possible be decreased by increasing 2008/3/410Shi-Chung Chang, NTUEE, GIIE, GICE

11. Q : (1) any limitations on increasing ? (2) If there are more than one component of r < 0 => which should we change to increase? Simplex method chooses one entering variable, Normally one with the most negative Q: which component should leave? the first one to reach 0 => satisfied again. 2008/3/411Shi-Chung Chang, NTUEE, GIIE, GICE

12. If we consider bringing into the basis, i.e., as the entering variable => 2008/3/412Shi-Chung Chang, NTUEE, GIIE, GICE

13. Mathematically, let be the new value vector of => The ith component of is zero when i.e. when 2008/3/413Shi-Chung Chang, NTUEE, GIIE, GICE

14. To maintain feasibility, the smallest of these ratios for i where > 0 determines how large can be. Say, it happens at l th components of  The new bfs has and leaves the basis while enters R earranging, w e can get 2008/3/414Shi-Chung Chang, NTUEE, GIIE, GICE

15. Revised Simplex Algorithm( one iteration) Step 1: Giving is the basis B such that Step 2: Solve to get Step 3 : Calculate If r ≥ 0 => optimal solution achieved, STOP Find K = arg Step 4 : Solve for y Step 5 : Find If, => unbounded solution, STOP Step 6 : Update the basic solution Step 7 : Update the basis Return to step 2 2008/3/415Shi-Chung Chang, NTUEE, GIIE, GICE

16. How to Update Basis Note that in the simplex algorithm we need to solve (step 2) (step 4) Where B’s differ by only one column between any two subsequent iterations. => How to calculate B’s efficiently? 2008/3/416Shi-Chung Chang, NTUEE, GIIE, GICE

17. Recall (rank one update) 2008/3/417Shi-Chung Chang, NTUEE, GIIE, GICE

18. Note: => Product Form of the inverse(PFI) 2008/3/418Shi-Chung Chang, NTUEE, GIIE, GICE

19. Some Computational Aspects Let be the elementary matrices after K pivot iterations. => For large-scale problems, store ‘s as vector and update and y as followers: 2008/3/419Shi-Chung Chang, NTUEE, GIIE, GICE

20. What if is small in pivoting?  Use LU or QR decompositions in computing If B is the current basis Note that => upper Hessenberg 2008/3/420Shi-Chung Chang, NTUEE, GIIE, GICE

21. Using a sequence of elimination steps on H: Store QR-decomposition Do givens transformation on H The simplex method is theoretically an exponential algorithm In practice, 2(n + m) iterations, i.e., a linear function of (n + m) 2008/3/421Shi-Chung Chang, NTUEE, GIIE, GICE

22. Convergence and Degeneracy Convergence: If the objective function value strictly decreases after every iteration, the simplex algorithm never repeats a basis and it converges in a finite number of iterations. why? Degeneracy: If for a such that => x is a degenerate 2008/3/422Shi-Chung Chang, NTUEE, GIIE, GICE

23. Property: in the pivoting step, => i.e. no change in cost. => Problem: cycling among degenerate => no guarantee of convergence => need an anti-cycling algorithm 2008/3/423Shi-Chung Chang, NTUEE, GIIE, GICE

24. How to get an initial feasible solution? (p)  (PH1) subject to 2008/3/424Shi-Chung Chang, NTUEE, GIIE, GICE

25. (P) has a feasible solution in (PH1) Q: can solve (PH1) using revised simplex with as the initial feasible solution 2008/3/425Shi-Chung Chang, NTUEE, GIIE, GICE

26. Solving S-P by the Revised Simplex Subject to * Flow balance eq. corresponding to * is dropped due to redundancy. 2008/3/426Shi-Chung Chang, NTUEE, GIIE, GICE

27. Chapter 3 Duality § 3.1 Dual LP Consider (SLP) Subject to 2008/3/427Shi-Chung Chang, NTUEE, GIIE, GICE

28. Dual Viewpoint From the optimality condition, if optimal solution to SLP => corresponding partition of A matrix such that Note that in Simplex Method, we have

29. Dual Viewpoint (Cont.) Note that The ’ we are looking for must satisfy (3.5) for any The root of (3.5) comes from So, Note that ’ in (3.5) and (3.6) satisfies (3.7) and (3.8).

30. Dual Objective Function Now consider the objective function From inequality (3.8) that then for any bfs  ’ is the one that leads to the maximum value while satisfying (3.7) and (3.8) we have and from (3.9)

31. Dual SLP (SLP) Subject to 2008/3/431Shi-Chung Chang, NTUEE, GIIE, GICE (DSLP) Subject to