1. 1 System Planning 2013 Lecture 7: Optimization Appendix A Contents: –General about optimization –Formulating optimization problems –Linear Programming (LP) –Mixed Integer Linear Programming (MILP)

2. 2 Optimization – General Discipline in applied mathematics Used when: –Want to maximize or minimize something (e.g. profit or cost) under various conditions (e.g. generation capacities, transmission limits)

3. 3 Optimization – General Typical problems In what sequence should parts be produced on a machine in order to minimize the change-over time? How can a dress manufacturer lay out its patterns on rolls of cloth to minimize wasted material? How many elevators should be installed in a new office building to achieve an acceptable expected waiting time? What is the lowest-cost formula for chicken food which will provide required quantities of necessary minerals and other nutrients? What is the generation plan for the power plants during the next six hours?

4. 4 Optimization - General The general optimization problem: f(x) : objective function x : optimization variables (vector) g(x), h(x) : constraints (vectors) and : variabel limits (vector) Defines the feasible set

5. 5 Optimization – Formulation Formulation of optimization problems: 1.Describe the problem in words: Think through the problem 2.Define symbols Review the parameters and variables involved 3.Write the problem mathematically Translate the problem defined in words to mathematics

6. 6 Optimization – Formulation The problem: A manufacturer owns a number of factories. How should the manufacturer ship his goods to the customers??

7. 7 Optimization – Formulation 1.The problem in words: How should the merchandise be shipped in order to minimize the transportation costs? The following must be fulfilled: The factories cannot produce more than their production capacities. The customers must at least receive their demand

8. 8 2. Define symbols: Index and parameters: m factories, factory i has capacity a i n customers, customer j demand b j units of the commodity The transportation cost from factory i to customer j is c ij per unit Variables Let x ij, i = 1...m, j = 1...n denote the number of units of the commodity shipped from factory i to customer j Optimization – Formulation

9. 9 3. Mathematical formulation: Objective function Transportation cost: Constraints Production capacity: Demand: Variable limits: Optimization – Formulation

10. 10 The optimization problem: Optimization – Formulation

11. 11 LP – General Linear Programming problems (LP problems) Class of optimization problems with linear objective function and constraints All LP problems can be written as (standard form):

12. 12 LP – General LP problems will here be explained through examples (similar as in Appendix A) Will have to look at the following: –Extreme points –Slack variables –Non existing solution –Not active constraints –No finite solution or unbounded problems –Degenerated solution –Flat optimum –Duality –Solution methods

13. 13 LP – Example The problem:

14. 14 LP – Example Optimum in: x 1 +x 2  2 4x 1 +12x 2  12 x 1  0 x 2  0 z = 20 z = 30 z = 40 Objective: x 2 1 2 3 4 x 1 1234

15. 15 LP – Extreme points The “corners” of the feasible set are called extreme points Optimum is always reached in one or more extreme points! x 1 +x 2  2 4x 1 +12x 2  12 x 1  0 x 2  0 z = 20 z = 30 z = 40 x 2 1 2 3 4 x 1 1234

16. 16 LP – Slack variables LP problem on standard form: Introduce slack variables in order to write inequality constraints using equality

17. 17 LP – Slack variables Without slack variables: ( Ax  b ) With slack variables: ( Ax = b )

18. 18 Can happen that the constraints makes no solution feasible! (Bad formulation of problem) New constraint LP – Non existing solution

19. 19 B: Feasible set defined by (c) A: Feasible set defined by (a) & (b) x 1 +x 2  2 4x 1 +12x 2  12 x 1  0 x 2  0 x 2 1 2 3 4 x 1 1234 (a) (b) x 1 +x 2  1 (c) Feasible set is empty! LP – Non existing solution

20. 20 LP – Not active constraints Some constraints might not be active in optimum. New constraint

21. 21 x 2 x 1 1 2 3 4 1234 x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 z = 20 z = 30 z = 40 x 1 +x 2  7 Not active constraints Active constraints LP – Not active constraints

22. 22 LP – No finite solution If the constraints don’t limit the objective: | z|   New objective function

23. 23 x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 x 2 x 1 1 2 3 4 1234 z = -5 z = -4 z = -3 min z  -  LP – No finite solution

24. 24 LP – Degenerated solution More than one extreme point can be optimal. All linear combinations of these points are then also optimal New objective

25. 25 x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 x 2 1 2 3 4 1234 z = 30 z = 40 z = 50 The line between the extreme points are also optimal solutions x 1 LP – Degenerated solution

26. 26 LP – Flat optimum A number of extreme points can have almost the same objective function value. New objective

27. 27 LP – Flat optimum x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 x 2 1 2 3 4 1234 z = 30 z = 40 z = 50 x 1 z = 19.980 z = 19.995

28. 28 LP – Duality All LP problems (primal problem) have a corresponding dual problem are called dual variables Primal problem: Dual problem:

29. 29 LP – Duality Theorem (strong duality): If the primal problem has an optimal solution, then also the dual problem has an optimal solution and the objective values of these solutions are the same.

30. 30 LP – Duality Q: What does the dual variables describe? A: Dual variables = Marginal value of the constraint that the dual variable represents, i.e. how the objective function value changes when the right-hand side of the constraint changes One dual variable for each constraint!

31. 31 LP – Duality x 2 x 1 1 2 3 4 1234 x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 z = 20 z = 30 z = 40 1 2 x 1 +x 2  7 3 In optimum: 1 > 0 (active) 2 > 0 (active) 3 = 0 (not active)

32. 32 Dual variables Small changes in right-hand side  b  Change in objective function value  z: LP – Duality  z = T  b

33. 33 MILP – General Mixed Integer Linear Programming problems (MILP problems) Class of optimization problems with linear objective function and constraints Some variables can only take integer values

34. 34 MILP – Example Optimum in: x 1 +x 2  2 4x 1 +12x 2  12 x 1  0 x 2  0 Objective: x 2 1 2 3 4 x 1 1234 z = 20 z = 30 z = 40

35. 35 MILP – Solving Easy to implement integer variables. Hard to solve the problems. –Execution times can increase exponentially with the number of integer variables Avoid integer variables if possible! Special case of integer variables: Binary variables x  {0,1}

36. 36 MILP – Example Minimize the cost for buying ”something” Variable, x : Quantity of ”something”, x  0. Not constant cost per unit - cost function. For example: Discount when buying more than a specified quantity: Cost [SEK] Number of ”units” xbxb x1x1 x2x2 Split x into two different variables, x 1 and x 2. Observe that x 1 ≤ x b. Also note that both x 1 and x 2 are  0.

37. 37 MILP – Example Cheaper to buy in segment 2  Must force the problem to fill the first segment before entering segment 2. Can be performed by introducing a binary variable. Cost [SEK] Number of ”units” xbxb x1x1 x2x2

38. 38 MILP – Example Introduce the following constraints: s=0s=1 Cost [SEK] Number of ”units” xbxb x1x1 x2x2 Check if s=0 : Check if s=1 : where M is an arbitrarely large number. Let s be an integer variable indicating whether being in segment 1 or 2.

39. 39 LP – Solution methods Simplex: Returns optimum in extreme point (also when degenerated solution) Interior point methods: If degenerated solution: Returns solution between two extreme points

40. 40 LP – Solution methods x 1 +x 2  2 x 2  0 4x 1 +12x 2  12 x 1  0 x 2 1 2 3 4 1234 z = 30 z = 40 z = 50 Simplex Interior point method

41. 41 PROBLEM 11 - Solutions to LP problems Assume that readymade software is used to solve an LP problem. In which of the following cases do you get an optimal solution to the LP problem and in which cases do you have to reformulate the problem (or correct an error in the code). a) The problem has no feasible solution. b) The problem is degenerated. c) The problem does not have a finite solution.