Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity » Polynomial time algorithms » Intractability » NP-complete problems 2.Basic Properties of Linear Programming » Forms of LP » Basic feasible solutions » Geometry of LP 3.The Revised Simplex Method

Today 1.The R evised Simplex Algorithm » Basics » The algorithm » Getting an Initial Feasible Solution 2.Computational Aspects » Updating the basis » Convergence and Degeneracy » The Revised Simplex Method 3.Duality » Dual LP » The Duality Theorem » Complementary Slackness » The Dual Simplex Algorithm 2008/3/42Shi-Chung Chang, NTUEE, GIIE, GICE

Reading Assignments: 1.Sections , of [PaS 8 3] 2.Sections of [Lue 8 4] Next Time: sections 4.3,4.4 and Chapter 5 of [PaS 83] 2008/3/43Shi-Chung Chang, NTUEE, GIIE, GICE

§2.2 The Revised Simplex Method The simplex method for LP : G.B. Dantzig 1951 Key idea: Phase1: find a bfs of Note that an LP may not have a solution Example min s.t. 2008/3/44Shi-Chung Chang, NTUEE, GIIE, GICE

Phase 2 Allow one of the zero components of the bfs to become positive and force one of the original positive components to become zero. => How to pick “entering” and “leaving” component Cost Traditional form of the simplex method: Tableau => read by yourself Here we consider matrix form for conciseness of presentation and later developments. 2008/3/45Shi-Chung Chang, NTUEE, GIIE, GICE

SLP: s.t. Assume P(A)=m. Partition A=[B:D] B: m linearly indep. columns of A (Assume the first m cols.) (SLP) subject to (3.1) 2008/3/46Shi-Chung Chang, NTUEE, GIIE, GICE

If and a basic solution When from (3.1) (note that may not ≥0 ) (3.2) Substituting(3.2) into the cost function (3.3) Define (3.4) as the relative cost vector => To minimize Z, we need only adjust Q: How? 2008/3/47Shi-Chung Chang, NTUEE, GIIE, GICE

Example subject to =2 + =5 + =6 In matrix form. 2008/3/48Shi-Chung Chang, NTUEE, GIIE, GICE

 2008/3/49Shi-Chung Chang, NTUEE, GIIE, GICE

Define the vector of simplex multipliers Compute in two steps: Assume that is non degenerate If is optimal and it does not pay to increase If then the cost Z can possible be decreased by increasing 2008/3/410Shi-Chung Chang, NTUEE, GIIE, GICE

Q : (1) any limitations on increasing ? (2) If there are more than one component of r < 0 => which should we change to increase? Simplex method chooses one entering variable, Normally one with the most negative Q: which component should leave? the first one to reach 0 => satisfied again. 2008/3/411Shi-Chung Chang, NTUEE, GIIE, GICE

If we consider bringing into the basis, i.e., as the entering variable => 2008/3/412Shi-Chung Chang, NTUEE, GIIE, GICE

Mathematically, let be the new value vector of => The ith component of is zero when i.e. when 2008/3/413Shi-Chung Chang, NTUEE, GIIE, GICE

To maintain feasibility, the smallest of these ratios for i where > 0 determines how large can be. Say, it happens at l th components of  The new bfs has and leaves the basis while enters R earranging, w e can get 2008/3/414Shi-Chung Chang, NTUEE, GIIE, GICE

Revised Simplex Algorithm( one iteration) Step 1: Giving is the basis B such that Step 2: Solve to get Step 3 : Calculate If r ≥ 0 => optimal solution achieved, STOP Find K = arg Step 4 : Solve for y Step 5 : Find If, => unbounded solution, STOP Step 6 : Update the basic solution Step 7 : Update the basis Return to step /3/415Shi-Chung Chang, NTUEE, GIIE, GICE

How to Update Basis Note that in the simplex algorithm we need to solve (step 2) (step 4) Where B’s differ by only one column between any two subsequent iterations. => How to calculate B’s efficiently? 2008/3/416Shi-Chung Chang, NTUEE, GIIE, GICE

Recall (rank one update) 2008/3/417Shi-Chung Chang, NTUEE, GIIE, GICE

Note: => Product Form of the inverse(PFI) 2008/3/418Shi-Chung Chang, NTUEE, GIIE, GICE

Some Computational Aspects Let be the elementary matrices after K pivot iterations. => For large-scale problems, store ‘s as vector and update and y as followers: 2008/3/419Shi-Chung Chang, NTUEE, GIIE, GICE

What if is small in pivoting?  Use LU or QR decompositions in computing If B is the current basis Note that => upper Hessenberg 2008/3/420Shi-Chung Chang, NTUEE, GIIE, GICE

Using a sequence of elimination steps on H: Store QR-decomposition Do givens transformation on H The simplex method is theoretically an exponential algorithm In practice, 2(n + m) iterations, i.e., a linear function of (n + m) 2008/3/421Shi-Chung Chang, NTUEE, GIIE, GICE

Convergence and Degeneracy Convergence: If the objective function value strictly decreases after every iteration, the simplex algorithm never repeats a basis and it converges in a finite number of iterations. why? Degeneracy: If for a such that => x is a degenerate 2008/3/422Shi-Chung Chang, NTUEE, GIIE, GICE

Property: in the pivoting step, => i.e. no change in cost. => Problem: cycling among degenerate => no guarantee of convergence => need an anti-cycling algorithm 2008/3/423Shi-Chung Chang, NTUEE, GIIE, GICE

How to get an initial feasible solution? (p)  (PH1) subject to 2008/3/424Shi-Chung Chang, NTUEE, GIIE, GICE

(P) has a feasible solution in (PH1) Q: can solve (PH1) using revised simplex with as the initial feasible solution 2008/3/425Shi-Chung Chang, NTUEE, GIIE, GICE

Solving S-P by the Revised Simplex Subject to * Flow balance eq. corresponding to * is dropped due to redundancy. 2008/3/426Shi-Chung Chang, NTUEE, GIIE, GICE

Chapter 3 Duality § 3.1 Dual LP Consider (SLP) Subject to 2008/3/427Shi-Chung Chang, NTUEE, GIIE, GICE

Dual Viewpoint From the optimality condition, if optimal solution to SLP => corresponding partition of A matrix such that Note that in Simplex Method, we have

Dual Viewpoint (Cont.) Note that The ’ we are looking for must satisfy (3.5) for any The root of (3.5) comes from So, Note that ’ in (3.5) and (3.6) satisfies (3.7) and (3.8).

Dual Objective Function Now consider the objective function From inequality (3.8) that then for any bfs  ’ is the one that leads to the maximum value while satisfying (3.7) and (3.8) we have and from (3.9)

Dual SLP (SLP) Subject to 2008/3/431Shi-Chung Chang, NTUEE, GIIE, GICE (DSLP) Subject to