1. EML 4550 - Spring’08 EML4552 - Engineering Design Systems II (Senior Design Project) Optimization Theory and Optimum Design Linear Programming Hyman: Chapter 10

2. EML 4550 - Spring’08 Optimization zMinimize (Maximize) an Objective Function of certain Variables subject to Constraints

3. EML 4550 - Spring’08 Linear Programming zMaximize a Linear Objective Function of i positive variables subject to j Linear Constraints

4. EML 4550 - Spring’08 Linear Programming zAlmost all linear optimization problems can be reduced to the canonical (standard) form yMinimization, change sign on objective function yVariables that can be negative, define: yConstraint reversal, ‘slack’ and ‘surplus’ variables

5. EML 4550 - Spring’08 L.P.: Geometric Interpretation

6. EML 4550 - Spring’08 L.P.: Geometric Interpretation x1x1 x2x2 (1) (2) (3) (4) U= 0 24 6.5

7. EML 4550 - Spring’08 L.P.: The “Simplex” Algorithm zStandard method to solve linear programming optimization problems zGeometric interpretation of linear programming problems (simple example) zBased on principle that optimum will be ON a constraint (typically a “corner”)

8. EML 4550 - Spring’08 Simplex Algorithm: Example

9. EML 4550 - Spring’08 Simplex Algorithm: Example Add x 4 to make the Inequality an equality

10. EML 4550 - Spring’08 Simplex Algorithm: Example zThe Simplex “Algorithm” consists of a number of steps carried out in ‘cycles’ until an optimum is reached z“Geometrically”, the Simplex method explores the objective function by ‘walking’ along the constraints in the ‘steepest’ direction until it reaches the optimum zBased on: yUnique solution yOptimum on a constraint

11. EML 4550 - Spring’08 Simplex Algorithm: Example zCycle 1 ySelect a starting point (a ‘corner’) yIn this example a corner is set by 3 variables (the other 3 can be solved from the constraints) zStep 1: Select the 3 ‘basis’ variables such that they appear in only one constraint and have coefficient of unity (x 4, x 5, x 6 in this case). Assign a value of 0 to the ‘non-basis’ variables. zStep 2:Determine coordinates of initial ‘corner’ (x 1 =0, x 2 =0, x 3 =0, x 4 =430, x 5 =460, x 6 =420)

12. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 3: Calculate objective function at the corner point (x 1 =0, x 2 =0, x 3 =0, x 4 =430, x 5 =460, x 6 =420), U=0 zStep 4: Move from corner to an ‘adjacent’ corner along the ‘edges’ represented by the constraints. Vary the non-basis variables to move along the edges. Move along the edge which will increase “U” the most. To do this, express the objective function in terms of the non-basis (U=3x 1 +2x 2 +5x 3, already, if not, solve the constraints to do so)

13. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 5: From the present ‘corner’ (x 1 =0, x 2 =0, x 3 =0) move along the variable that will produce the largest increase in U (highest coefficient), in this case x 3. We re-formulate the basis by introducing x 3 and deleting from the basis one of the variables. zStep 6: We want to increase x 3 by as much as possible without violating any constraint (i.e., move to the next ‘corner’). To do this, reformulate the constraints for the ‘incoming’ variable to the basis

14. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 6 (cont.): In this case for x 1 =x 2 =0 and the incoming x3, the constraints are: zStep 7: Determine how much x 3 can increase before driving any variable negative. Limits are: (a) 430, (b) 230, (c)  therefore, x 3 can be increased up to 230 and x 5 leaves the basis

15. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 8: x 3, x 4, and x 6 are the new basis, set the non-basis variables to zero, x 1 =0, x 2 =0, x 5 =0 zPrepare to start the next cycle (repeat procedure)

16. EML 4550 - Spring’08 Simplex Algorithm: Example zCycle 2 zStep 1: The new basis was chosen as the last step of the previous cycle (x 3, x 4, x 6 ). zStep 2:Determine coordinates of new ‘corner’ by re-writing constraints: (x 1 =0, x 2 =0, x 3 =230, x 4 =200, x 5 =0, x 6 =420)

17. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 3: Calculate objective function at the corner point (x 1 =0, x 2 =0, x 3 =230, x 4 =200, x 5 =0, x 6 =420), U=3(0)+2(0)+5(230)=1150 zStep 4: Prepare to choose new basis. Re-write U in terms of non-basis variables (using the constraints to eliminate the basis variables):

18. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 5: Move away from present corner along the edge that will increase U the most. From the objective function, only an increase in x 2 will yield improvement (all other coefficients are negative). Introduce x 2 to the basis. zStep 6: Re-write constraints in terms of new basis variable:

19. EML 4550 - Spring’08 Simplex Algorithm: Example zStep 7: Determine how much x 2 can increase before driving any variable negative. Limits are: (a) 100 (for x 3 =230), (b) , (c) 105 therefore, x 2 can be increased up to 100 and x4 leaves the basis zStep 8: x 2, x 3, and x 6 are the new basis, set the non-basis variables to zero, x 1 =0, x 4 =0, x 5 =0 zPrepare to start the next cycle (repeat procedure)

20. EML 4550 - Spring’08 Simplex Algorithm: Example zCycle 3 zRepeat the same steps now with x 2, x 3, x 6 as the basis. After steps 2 and 3 the solution is x 1 =0, x 2 =100, x 3 =230, x 4 =0, x 5 =0, x 6 =20, U=1350 (further improvement). At step 4, however, all coefficients on the objective function, U, are negative, so no further improvements are possible, we are done! We have found the optimum (x 1 =0, x 2 =100, x 3 =230, U=1350).

21. EML 4550 - Spring’08

22. Simplex Algorithm zFound the optimum with exploration of just 3 ‘corners’ zEfficiency increases tremendously with the complexity of the problem (number of variables and constraints) z“New” L.P. algorithms may be even more efficient in extremely large problems