1. Discrete Optimization
MA2827
Fondements de l’optimisation discrète
Material from P. Van Hentenryck’s course

2. Outline Linear Programming Mixed Integer Program
Examples (TSP, Knapsack)

3. What is a linear program?

4. What is a linear program?
n variables, m constraints
Variables are non-negative
Inequality constraints

5. What is a linear program?
What about maximization?
solve
What if a variable can take negative values?
replace xi by
Equality constraints?
use two inequalities
Variables taking integer values?
mixed integer programming

6. Geometrical view
First, convex sets

7. Geometrical view
First, convex sets

8. Geometrical view
Convex combinations

9. Geometrical view Convex sets The intersection of convex sets is convex
A set S in Rn is convex if it contains all the convex combinations of the points in S
The intersection of convex sets is convex
Proof ?

10. Geometrical view A half space is a convex set
Polyhedron: intersection of set of half spaces (also convex). If finite, polytope

11. Geometrical view

12. Geometrical view

13. Geometrical view

14. Geometrical view

15. Geometrical view

16. Geometrical view

17. Geometrical view

18. Geometrical view

19. Geometrical view

20. Geometrical view

21. Geometrical view

22. Geometrical view

23. Geometrical view

24. Geometrical view

25. Geometrical view

26. Geometrical view

27. Geometrical view

28. Geometrical view

29. Geometrical view

30. Geometrical view

31. Geometrical view

32. Geometrical view

33. Geometrical view

34. Geometrical view

35. Geometrical view

36. Geometrical view

37. Geometrical view

38. Geometrical view

39. Geometrical view

40. Geometrical view

41. Geometrical view

42. Geometrical view

43. Geometrical view
Theorem: At least one of the points where the objective value is minimal is a vertex
Proof ?

44. Geometrical view

45. Geometrical view How to solve a linear program?
Enumerate all the vertices
Select the one with the smallest objective value

46. Algebraic view The simplex algorithm Invented by G. Dantzig
A more intelligent way of exploring the vertices
Invented by G. Dantzig
Exponential worst-case, but works well in practice

47. Simplex algorithm Outline An optimal solution is at a vertex
A vertex is a basic feasible solution (BFS)
You can move from one BFS to a neighboring BFS
You can detect whether a BFS is optimal
From any BFS, you can move to a BFS with better a cost

48. Basic feasible solution (BFS)

49. Basic feasible solution (BFS)

50. Basic feasible solution (BFS)

51. Basic feasible solution (BFS)

52. Basic feasible solution (BFS)

53. Basic feasible solution (BFS)

54. Basic feasible solution (BFS)

55. Basic feasible solution (BFS)

56. Basic feasible solution (BFS)

57. Basic feasible solution (BFS)

58. Basic feasible solution (BFS)
Select m variables
Will be the basic variables
Re-write them using only non-basic variables
Gaussian elimination
If all the b’s are non-negative
We have a BFS

59. Basic feasible solution (BFS)
But we do not have equalities

60. Basic feasible solution (BFS)
But we do not have equalities
Slack variables

61. Basic feasible solution (BFS)
Re-write the constraints as equalities
With slack variables
Select m variables
Will be the basic variables
Re-write them using only non-basic variables
Gaussian elimination
If all the b’s are non-negative
We have a BFS

62. Naïve algorithm Generate all basic feasible solutions
i.e., select m basic variables, perform Gaussian elim.
test whether it is feasible
Select the BFS with the best cost
But,
Can we explore this space more efficiently?

63. Simplex algorithm Outline An optimal solution is at a vertex
A vertex is a basic feasible solution (BFS)
You can move from one BFS to a neighboring BFS
You can detect whether a BFS is optimal
From any BFS, you can move to a BFS with better a cost

64. Simplex algorithm Local search algorithm Move from BFS to BFS
Guaranteed to find a global optimum
Due to convexity
Key idea: How do we do this move operation?

65. Simplex algorithm

66. Simplex algorithm Move to another BFS (local search)
Select a non-basic variable with negative coeff. (entering variable)
Replace a basic variable with this selection (leaving variable)
Perform Gaussian elimination

67. Simplex algorithm

68. Simplex algorithm

69. Simplex algorithm

70. Simplex algorithm

71. Simplex algorithm

72. Simplex algorithm

73. Simplex algorithm

74. Simplex algorithm Move to another BFS: Select the entering variable xe
Non-basic variable with negative coeff.
Select the leaving variable xl to maintain feasibility
Apply Gaussian elimination
i.e., eliminate xe from the the right hand side
Pivot(e,l)

75. Simplex algorithm Outline An optimal solution is at a vertex
A vertex is a basic feasible solution (BFS)
You can move from one BFS to a neighboring BFS
You can detect whether a BFS is optimal
From any BFS, you can move to a BFS with better a cost

76. Simplex algorithm
A BFS is optimal if the objective function, after eliminating all the basic variables is

77. Simplex algorithm
Example

78. Simplex algorithm
Example

79. Simplex algorithm

80. Simplex algorithm

81. Simplex algorithm

82. Simplex algorithm

83. Simplex algorithm

84. Simplex algorithm Outline An optimal solution is at a vertex
A vertex is a basic feasible solution (BFS)
You can move from one BFS to a neighboring BFS
You can detect whether a BFS is optimal
From any BFS, you can move to a BFS with better a cost

85. Simplex algorithm Local move improves the objective value
In a BFS, assume that
b1 > 0, b2 > 0, …, bm > 0
there exists an entering variable e with ce < 0
Then, the move pivot(e,l) is improving

86. Simplex algorithm If Algorithm terminates with an optimal solution
b1, b2, …, bm are strictly positive
objective function is bounded below
Algorithm terminates with an optimal solution

87. Simplex algorithm
Selecting the leaving variable
No leaving variable!

88. Simplex algorithm Basic solution What if I increase the value of x1
Solution remains feasible, but
Value of objective function decreases

89. Simplex algorithm Another issue: What if some bi becomes zero?
Leaving variable: x2

90. Simplex algorithm Outline An optimal solution is at a vertex
A vertex is a basic feasible solution (BFS)
You can move from one BFS to a neighboring BFS
You can detect whether a BFS is optimal
From any BFS, you can move to a BFS with better a cost
Need to find a new way to guarantee termination

91. Simplex algorithm Termination
Bland rule: select always the first entering variable lexicographically
Lexicographic pivoting rule: break ties when selecting the leaving variable
Pertubation methods

92. Simplex algorithm How do I find my first BFS ?
Introduce artificial variables

93. Simplex algorithm
Easy BFS
But wrt another problem

94. Simplex algorithm Two-phase strategy First find a BFS (if one exists)
Then, find optimal BFS

95. Simplex algorithm Find a BFS Feasible if the objective value is 0
i.e., all yi’s are 0, and there is a BFS without yi (almost always)

96. Simplex algorithm