1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Lecture 8 Tuesday, 11/19/02 Linear Programming

2. Overview ä Motivation & Basics ä Standard & Slack Forms ä Formulating Problems as Linear Programs ä Simplex Algorithm ä Duality ä Initial Basic Feasible Solution ä Literature Case Study

3. Motivation & Basics

4. Motivation: A Political Problem source: 91.503 textbook Cormen et al. Goal: Win election by winning majority of votes in each region. Subgoal: Win majority of votes in each region while minimizing advertising cost. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads

5. Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural source: 91.503 textbook Cormen et al.

6. General Linear Programs real numbers variables Linear function Linear inequalities Linear constraints source: 91.503 textbook Cormen et al.

7. Overview of Linear Programming Convex feasible region Objective function Objective value source: 91.503 textbook Cormen et al.

8. Standard & Slack Forms

9. Standard Form objective function constraints source: 91.503 textbook Cormen et al.

10. Standard Form (compact) mxn matrix m-dimensional vector n-dimensional vectors source: 91.503 textbook Cormen et al. Can specify linear program in standard form by (A,b,c).

11. Converting to Standard Form source: 91.503 textbook Cormen et al.

12. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Negate coefficients Transforming minimization to maximization

13. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. If x j has no non-negativity constraint, replace each occurrence of x j with x j ’ – x j ”. Giving each variable a non-negativity constraint New non-negativity constraints

14. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Transforming equality constraints to inequality constraints

15. Converting to Standard Form (continued) source: 91.503 textbook Cormen et al. Changing sense of an inequality constraint Rationale:

16. Converting Linear Programs into Slack Form source: 91.503 textbook Cormen et al. for algorithmic ease, transform all constraints except non-negativity ones into equalities for inequality constraint: define slack slack variable instead of s basic variables non-basic variables

17. Converting Linear Programs into Slack Form (continued) source: 91.503 textbook Cormen et al. objective function

18. Converting Linear Programs into Slack Form (continued) source: 91.503 textbook Cormen et al. Compact Form: (N, B, A, b, c, v) set of indices of non-basic variables set of indices of basic variables Slack Form Example Compact Form negative of slack form coefficients

19. Formulating Problems as Linear Programs

20. Shortest Paths source: 91.503 textbook Cormen et al. Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not?

21. Maximum Flow source: 91.503 textbook Cormen et al.

22. Minimum Flow source: 91.503 textbook Cormen et al.

23. Multicommodity Flow source: 91.503 textbook Cormen et al. should be s i

24. Simplex Method

25. Solving a Linear Program source: 91.503 textbook Cormen et al. ä Simplex algorithm ä Geometric interpretation ä Visit vertices on the boundary of the simplex representing the convex feasible region ä Transforms set of inequalities using process similar to Gaussian elimination ä Run-time ä not polynomial in worst-case ä often very fast in practice ä Ellipsoid method ä Run-time ä polynomial ä slow in practice ä Interior-Point methods ä Run-time ä polynomial ä for large inputs, performance can be competitive with simplex method ä Moves through interior of feasible region

26. Simplex Algorithm: Example Basic Solution source: 91.503 textbook Cormen et al. Standard Form Slack Form Basic Solution: set each nonbasic variable to 0. Basic Solution:

27. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x 1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x 6 of that constraint, swap role with x 1. Rewrite other equations with x 6 on RHS. PIVOT leaving variable entering variable new objective value

28. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Next Iteration: select x 3 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

29. Simplex Algorithm: Example Reformulating the LP Model source: 91.503 textbook Cormen et al. Next Iteration: select x 2 as entering variable. PIVOT leaving variable entering variable New Basic Solution: new objective value

30. Simplex Algorithm: Issues to be Addressed… source: 91.503 textbook Cormen et al.

31. Simplex Algorithm: Pivoting source: 91.503 textbook Cormen et al. leaving variable entering variable Rewrite the equation that has x l on LHS to have x e on LHS Update remaining equations by substituting RHS of new equation for each occurrence of x e. Do the same for objective function. Update sets of nonbasic, basic variables.

32. Simplex Algorithm: Pivoting (continued) source: 91.503 textbook Cormen et al.

33. Simplex Algorithm: Pseudocode source: 91.503 textbook Cormen et al. to be defined later (detects infeasibility) Feasibility initial basic solution optimal solution

34. Simplex Algorithm: Correctness Proof Strategy source: 91.503 textbook Cormen et al. ä If SIMPLEX has initial feasible solution and eventually terminates, then it either ä returns a feasible solution or ä determines that the LP is unbounded. ä SIMPLEX terminates. ä SIMPLEX’s solution is optimal.

35. Simplex Algorithm: Correctness Proof Approach, Part I source: 91.503 textbook Cormen et al. ä If SIMPLEX has initial feasible solution and eventually terminates, then it either ä returns a feasible solution or ä determines that the LP is unbounded.

36. Simplex Algorithm: Correctness Proof Approach, Part II source: 91.503 textbook Cormen et al.

37. Duality Simplex Algorithm: Correctness Proof Approach, Part III

38. Linear Programming Duality source: 91.503 textbook Cormen et al. max becomes min RHS coefficients swap places with objective function coefficients sense changes x variables go away y variables appear

39. Duality Example source: 91.503 textbook Cormen et al.

40. Weak Linear Programming Duality source: 91.503 textbook Cormen et al. Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP.

41. Weak Linear Programming Duality (continued) source: 91.503 textbook Cormen et al.

42. Finding a Dual Solution source: 91.503 textbook Cormen et al. Finding a dual solution whose value is equal to that of an optimal primal solution…

43. Optimality source: 91.503 textbook Cormen et al.

44. Initial Basic Feasible Solution

45. Finding an Initial Solution source: 91.503 textbook Cormen et al. An LP model whose initial basic solution is not feasible

46. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. Auxiliary LP model L aux :

47. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al.

48. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. Original LP model L aux L aux in slack form

49. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al. PIVOT

50. Finding an Initial Solution (continued) source: 91.503 textbook Cormen et al.

51. Fundamental Theorem of Linear Programming source: 91.503 textbook Cormen et al.

52. Literature Case Study