Integer Programming An integer linear program (ILP) is defined exactly as a linear program except that values of variables in a feasible solution have to be integral: max 𝑐 ⊤ 𝑥 s.t. 𝐴𝑥≤𝑏 𝑥≥0 𝑥∈ ℤ 𝑛 Very useful new possibility: We can now model Boolean variables. This makes ILPs much more powerful than LPs for modelling purposes! Unfortunately it also makes them a lot harder to solve computationally!

Integer Linear Program in standard form P: max 𝑐 ⊤ 𝑥 s.t. 𝐴𝑥≤𝑏 𝑥≥0 𝑥∈ ℤ 𝑛 where 𝐴∈ ℝ 𝑚×𝑛 , 𝑏∈ ℝ 𝑚 ,c∈ ℝ 𝑛 .

Bounded Integer Linear Program P: max 𝑐 ⊤ 𝑥 s.t. 𝐴𝑥≤𝑏 𝑙 𝑖 ≤𝑥 𝑖 ≤ 𝑢 𝑖 𝑥∈ ℤ 𝑛 Where 𝐴∈ ℝ 𝑚×𝑛 , 𝑏∈ ℝ 𝑚 ,c∈ ℝ 𝑛 and 𝑙 𝑖 < 𝑢 𝑖 , 𝑙 𝑖 , 𝑢 𝑖 ∈ℤ.

Branch and Bound Solve P* (using e.g. Simplex algorithm) If P* is infeasible, then P is infeasible (return (𝐧𝐮𝐥𝐥,−∞)). Otherwise let ( 𝑥 ∗ , 𝜆 ∗ ) be optimal solution to P*. If 𝑥 ∗ is integer, return ( 𝑥 ∗ , 𝜆 ∗ ) . If 𝜆 ∗ ≤currentbest (bound), then return from branch with no solution (return (𝐧𝐮𝐥𝐥,−∞)). Otherwise, pick 𝑖 such that 𝑥 𝑖 ∗ ∉ℤ. Recursively solve (branch) (P*[ 𝑢 𝑖 ≔ 𝑥 𝑖 ∗ ], currentbest) to get ( 𝑥 𝑙 , 𝜆 𝑙 ). If 𝜆 𝑙 >currentbest, update currentbest≔ 𝜆 𝑟 Recursively solve (branch) (P*[ 𝑙 𝑖 ≔ 𝑥 𝑖 ∗ ], currentbest) to get ( 𝑥 𝑟 , 𝜆 𝑟 ). If 𝜆 𝑙 > 𝜆 𝑟 , return ( 𝑥 𝑙 , 𝜆 𝑙 ), otherwise return ( 𝑥 𝑟 , 𝜆 𝑟 ).

Termination If 𝑢 𝑖 = 𝑙 𝑖 for all 𝑖 we terminate immediately. If we do not terminate, then in each recursive call, either one 𝑙 𝑖 increased or one 𝑢 𝑖 has decreased. Since 𝑙 𝑖 and 𝑢 𝑖 are integers, we terminate.

Complexity Unit of time = number of leaves in the recursion tree. We can show by induction: 𝑇≤ 𝑖 𝑛 𝑢 𝑖 − 𝑙 𝑖 +1 . In practice, it can be hoped that many subcalls finish early. However, all known algorithms for solving integer linear programs are worst case exponential time. In contrast to the case of the Simplex algorithm, the worst case is no longer exceptional!

Non-depth first Branch and Bound Instead of letting the recursion stack represent the branch and bound tree, we can make it an explicit data structure This allows us to do best first search: Choose to expand the subproblem with the lowest relaxed solution value. Hope: This also contains very good valid solutions which will allow the currentbest value to decrease significantly.

Branch and Bound activeset := root currentbest := ∞; ​while activeset is not empty do choose a branching node, node 𝑘∈ activeset remove node 𝑘 from activeset; generate the subproblems of node 𝑘, 𝑖=1,…, 𝑛 𝑘 and the corresponding relaxed optimal values 𝜆 1 ,…, 𝜆 𝑛 𝑘 . ​for 𝑖=1,…, 𝑛 𝑘 do ​if 𝑧 𝑖 ≥ currentbest then kill child 𝑖 ​else if optimal relaxed solution of child 𝑖 is integral then currentbest := 𝜆 𝑖 ​else add child 𝑖 to activeset

Branch and Bound in a general context Ingredients needed: Relaxation method. Expands space of feasible solution. Relaxed instance must be efficiently solvable. Search space division. The optimal solution of the relaxed problem can not be feasible in any of the relaxed subproblems (it should be “killed”) Any non-relaxed feasible solution should be feasible in one of the subproblems.

Traveling Salesman Problem (Section 23.2) Input: Cost (distance) matrix 𝐶=( 𝑐 𝑖𝑗 ), 𝑖,𝑗=1,…,𝑛 Output: Permutation 𝜋 on {0,1,…,𝑛−1} minimizing 𝑖=1 𝑛 𝑐 𝜋(𝑖),(𝜋 𝑖+1 mod 𝑛) .

State of the art for TSP TSP instances of thousand of cities can be consistently solved to optimality. Instances of up to 25000 cities have been solved: All cities in Sweden! Technique: Branch-and-Bound combined with cutting plane algorithms: Branch-and-cut.

Valid inequality for an ILP Given integer linear program, a new inequality is called valid if it does not change the set of integer solutions to the program. It is a cutting plane if it removes the (non-integer) optimum solution to the relaxed linear program.

Cutting plane algorithm for ILP input ILP instance P P’ := LP-relaxation of P while(optimal solution to P’ is not integer) { add valid cutting plane to P P’ := LP-relaxation of P } return optimal solution to P’ How to find valid cutting plane?

Gomory Cutting Plane Algorithm Assume that all initial coefficients in (standard form) ILP instance are integer. Solve LP-instance using simplex method. Suppose the simplex method terminates with a non-integer optimal solution.

Suppose some line in final dictionary reads 𝑥 𝑖 = 𝑏 𝑖 + 𝑗 𝑎 𝑗 𝑥 𝑗 where 𝑏 𝑖 is not integer. Let 𝑏 𝑖 ′ = 𝑏 𝑖 − 𝑏 𝑖 and 𝑎 𝑗 ′ = 𝑎 𝑗 − 𝑎 𝑗 . Then 𝑥 𝑖 − 𝑏 𝑖 − 𝑗 𝑎 𝑗 𝑥 𝑗 = 𝑏 𝑖 ′ + 𝑗 𝑎 𝑗 ′ 𝑥 𝑗 For every feasible integer solution: Left hand side is integer. Right hand side is strictly bigger than zero. Thus 𝑥 𝑖 − 𝑏 𝑖 − 𝑗 𝑎 𝑗 𝑥 𝑗 ≥1 is a valid cutting plane.

Gomory Cutting Plane Algorithm Partial correctness is obvious. Termination is not guaranteed. Can be guaranteed if simplex algorithm uses special pivot rule and line of final dictionary is chosen carefully (non-trivial!) Not very practical by itself, but is component of state-of-the-art solvers.

Cutting planes for TSP (and other problems) Use ILP formulation of TSP. Use specially tailored cutting planes.

Traveling Salesman Problem - ILP formulation Decision variables: 𝑥 𝑖𝑗 ∈ 0,1 , 𝑖,𝑗=1,…,𝑛 𝑡 𝑗 ∈{0,1,…,𝑛−1}, j=0,…,𝑛 Interpretation: 𝑥 𝑖𝑗 =1 if and only if for some 𝑘 it holds 𝜋 𝑘 =𝑖 and 𝜋( 𝑘+1 mod 𝑛)=j 𝑡 𝑗 =𝑖 if and only if 𝜋(𝑖)=𝑗

Traveling Salesman Problem - ILP formulation min 𝑖,𝑗=1 𝑛 𝑐 𝑖𝑗 𝑥 𝑖𝑗 s.t. 𝑗=1 𝑛 𝑥 𝑖𝑗 =1 , for 𝑖=1,…,𝑚 𝑖=1 𝑛 𝑥 𝑖𝑗 =1 , for 𝑗=1,…,𝑚 𝑡 𝑗 ≥ 𝑡 𝑖 +1−𝑛 1− 𝑥 𝑖𝑗 , for 𝑖≥0, 𝑗≥1,𝑖≠𝑗

Symmetric TSP Distance from i to j is equal to distance from j to i. Let 𝑦 𝑖𝑗 = 𝑥 𝑖𝑗 + 𝑥 𝑗𝑖 Consider a cut (𝑆,𝑇), 𝑆∩𝑇=∅, 𝑆∪𝑇={1,…,𝑛} The TSP tour has to cross the cut 𝑆,𝑇 at least twice! 𝑖∈𝑆 𝑗∈𝑇 𝑦 𝑖𝑗 ≥2 is a valid inequality.

Cutting plane algorithm for TSP Find optimal solution 𝑥 ∗ of relaxaton 𝑃 ∗ . Let 𝑦 𝑖𝑗 = 𝑥 𝑖𝑗 + 𝑥 𝑗𝑖 Check if some 𝑆,𝑇 has 𝑖∈𝑆, 𝑗∈𝑇 𝑦 𝑖𝑗 <2 . (this can be efficiently checked by Max Flow algorithm using the max-flow min-cut theorem). Add inequality 𝑖∈𝑆, 𝑗∈𝑇 𝑦 𝑖𝑗 ≥2 .

Cutting plane algorithm for TSP At some point we will fail to find new cuts. Still may not have integer solution. If not, add other kinds of specially tailored inequalities (big literature) or BRANCH.

Branch-and-cut for TSP Branch-and-bound with relaxation being LP- relaxation + some set of valid inequalities. When to stop adding inequalities and start branching is a matter of heuristics and experiments. Yields state-of-the art solver. Many non-trivial implementation issues.