The Problem with Integer Programming H.P.Williams London School of Economics

The Nature of Integer Programming (IP)  Is IP like Linear Programming (LP) ?  Applications of Integer Programming  Mathematical Properties of IP  Economic Properties of IP  Chvátal Functions and Integer Monoids

A General (Mixed) Integer Programme (IP) Maximise/Minimise ∑ j c j x j +∑ k d k y k Subject to : ∑ j a ij x j +∑ k e ik y k b i for all i x j >=0 all j, y k >=0 all k and integer x j >=0 all j, y k >=0 all k and integer

A General (Mixed) Integer Programme  Frequently (but not always) the integer variables are restricted to values 0 and 1 representing (indivisible) Yes/No decisions eg. Investment  Can view as a Logical statement about a series of Linear Programmes (LPs)  Leads a to close relationship between Logic and IP

0-1 Integer Programmes  Any IP with bounded integer variables can be converted to a 0-1 IP  0-1 IPs can be interpreted as Disjunctions of LPs of LPs  Application of logical methods to formulation and solution

Applications of IP  Extensions to LPs eg Manufacturing, Distribution, Petroleum, Gas and Chemicals  Global Optimisation of non-convex (non-linear) models  Power Systems Loading  Facilities Location  Routing  Telecommunications  Medical Radiation  Statistical Design  Molecular Biology  Genome Sequencing  Archaeological Seriation  Optimal Logical Statements  Computer Design  Aircraft Scheduling  Crew Rostering

Linear Programming v Integer Programming  An LP Minimise X 2 Subject to: 2X 1 +X 2 >=13 5X 1 +2X 2 <=30 5X 1 +2X 2 <=30 -X 1 +X 2 >=5 -X 1 +X 2 >=5 X 1, X 2 >= 0 X 1, X 2 >= 0 The Solution  X 1 = 2 2 /3, X 2 = 7 2 /3

Linear Programming v Integer Programming  An LP Minimise X 2 Subject to: 2X 1 +X 2 >=13 5X 1 +2X 2 <=30 5X 1 +2X 2 <=30 -X 1 +X 2 >=5 -X 1 +X 2 >=5 X 1, X 2 >= 0 X 1, X 2 >= 0 The Solution  X 1 = 2 2 /3, X 2 = 7 2 /3  An IP Minimise X 2 Subject to: 2X 1 + X 2 >=13 5X 1 + 2X 2 <=30 5X 1 + 2X 2 <=30 -X 1 + X 2 >=5 -X 1 + X 2 >=5 X 1, X 2 >= 0 and integer

Linear Programming v Integer Programming  An LP Minimise X 2 Subject to: 2X 1 +X 2 >=13 5X 1 +2X 2 <=30 5X 1 +2X 2 <=30 -X 1 +X 2 >=5 -X 1 +X 2 >=5 X 1, X 2 >= 0 X 1, X 2 >= 0 The Solution  X 1 = 2 2 /3, X 2 = 7 2 /3 The Solution  X 1 = 2, X 2 = 9  An IP Minimise X 2 Subject to: 2X 1 + X 2 >=13 5X 1 + 2X 2 <=30 5X 1 + 2X 2 <=30 -X 1 + X 2 >=5 -X 1 + X 2 >=5 X 1, X 2 >= 0 and integer

LP and IP Solutions Min x Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 x 2 -x 1 + x 2 >= 5 x 2 -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= x x 1

LP and IP Solutions 9 Optimal IP Solution (2, 9). Min x 2 9 Optimal IP Solution (2, 9). Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= 5 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= 0 x 2 x x x 1

IP Solution after removing constraint 1 Min x 2 Min x 2 8 c1... c 3 st 5x 1 + 2x 2 <= 30 8 c1... c 3 st 5x 1 + 2x 2 <= 30 -x 1 + x 2 >= 5 -x 1 + x 2 >= x 1, x 2 >= x 1, x 2 >= 0 x c c 2 Optimal IP Solution (0, 5) Optimal IP Solution (0, 5) x x 1

IP Solution 9 Optimal IP Solution (2, 9). Min x 2 9 Optimal IP Solution (2, 9). Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= 5 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= 0 x 2 x x x 1

IP Solution after removing constraint Min x Min x 2 c1 c3 st 2x 1 + x 2 >= 13 c1 c3 st 2x 1 + x 2 >= Optimal IP Solution (3, 8) Optimal IP Solution (3, 8) -x 1 + x 2 >= 5 -x 1 + x 2 >= x 1, x 2 >= x 1, x 2 >= 0 x x x

IP Solution 9 Optimal IP Solution (2, 9). Min x 2 9 Optimal IP Solution (2, 9). Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= 5 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= 0 x 2 x x x

IP Solution after removing constraint Min x Min x 2 st 2x 1 + x 2 >= 13 st 2x 1 + x 2 >= x 1 + 2x 2 <= x 1 + 2x 2 <= 30 c1 c2 x 1, x 2 >= 0 c1 c2 x 1, x 2 >= x 2 x Optimal IP Solution (4, 5) Optimal IP Solution (4, 5) x x 1

Rounding often not satisfactory Example: The Alabama Paradox  State Population Fair solution Rounded Solution With 10 Representatives With 10 Representatives A 621k A 621k B 587k B 587k C 201k C 201k With 11 Representatives With 11 Representatives A 621k A 621k B 587k B 587k C 201k C 201k

IP Formulation of Political Apportionment Problem V i = Population (Votes cast) for State (Party) i V i = Population (Votes cast) for State (Party) i x i = Seats allotted to State (Party) i x i = Seats allotted to State (Party) i Choose x i so as to: Choose x i so as to: Min Max i (x i / v i ) Min Max i (x i / v i ) st ∑ i x i = Total Number of Seats st ∑ i x i = Total Number of Seats x i >= 0 and integer for all I x i >= 0 and integer for all I ie Min y ie Min y st x i / v i <= y for all i st x i / v i <= y for all i ∑ i x i = Total Number of Seats ∑ i x i = Total Number of Seats x i >= 0 and integer for all I x i >= 0 and integer for all I LP Relaxation gives fractional solution LP Relaxation gives fractional solution IP Solution give Jefferson/D’Hondt solution IP Solution give Jefferson/D’Hondt solution

IP Solution  State Population Fractional Rounded Jefferson/ solution solution D’Hondt solution solution solution D’Hondt solution (LP) (IP) (LP) (IP) With 10 Representatives With 10 Representatives A 621k A 621k B 587k B 587k C 201k C 201k With 11 Representatives With 11 Representatives A 621k A 621k B 587k B 587k C 201k C 201k

Mathematical Differences between LP and IP  Consider a (Pure) IP in standard form Maximise c 1 x 1 + c 2 x 2 + … + c n x n Subject to: a 11 x 1 + a 12 x 2 + … a 1n x n <= b 1 a 21 x 1 + a 22 x 2 + … a 2n x n <= b 2 a 21 x 1 + a 22 x 2 + … a 2n x n <= b 2.. a m1 x 1 + a m2 x 2 + … a mn x n <= b m a m1 x 1 + a m2 x 2 + … a mn x n <= b m x 1, x 2, …, x n >= 0 and integer x 1, x 2, …, x n >= 0 and integer

Mathematical Differences between LP and IP  LP IP If has Optimal Solution No limit on number of positive variables If has Optimal Solution No limit on number of positive variables there is one with at there is one with at most m variables positive most m variables positive (a basic solution) Hilbert Basis (no fixed dimension) (a basic solution) Hilbert Basis (no fixed dimension) At most n constraints At most 2 n – 1 At most n constraints At most 2 n – 1 binding at optimum constraints binding binding at optimum constraints binding at optimum at optimum There are valuations Chvátal Functions There are valuations Chvátal Functions on constraints which on constraints which close duality gap ie close duality gap ie there is a (symmetric) LP No obvious symmetry there is a (symmetric) LP No obvious symmetry (dual) model (dual) model

IPs involve Lattices within Polytopes Eg Max 2x 1 +x 2 st 2x 1 +9x 2 <=80 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 -x 1 <=0 -x 1 <=0 -x 2 <=0 -x 2 <=0 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) x 1 ≡0(mod1) x 1 ≡0(mod1) x 2 ≡0(mod1) x 2 ≡0(mod1)

What are the strongest implications? Max 2x 1 +x 2 st 2x 1 +9x 2 <=80 2x 1 +9x 2 <=80 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 -x 1 <=0 -x 1 <=0 2x 1 +x 2 ? -x 1 <=0 -x 1 <=0 2x 1 +x 2 ? -x 2 <=0 -x 2 <=0 -x 2 <=0 -x 2 <=0 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) x 1 ≡0(mod1) x 1 ≡0(mod1) 2x 1 +x 2 ? x 1 ≡0(mod1) x 1 ≡0(mod1) 2x 1 +x 2 ? x 2 ≡0(mod1) x 2 ≡0(mod1) x 2 ≡0(mod1) x 2 ≡0(mod1)

What are the strongest implications? Dual arguments. Max 2x 1 +x 2 st 2x 1 +9x 2 <=80 2x 1 +9x 2 <=80 ⅓ 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 ⅔ 2x 1 -3x 2 <=6 2x 1 -3x 2 <=6 ⅔ -x 1 <=0 -x 1 <=0 0 2x 1 +x 2 <= 30 2 /3 -x 1 <=0 -x 1 <=0 0 2x 1 +x 2 <= 30 2 /3 -x 2 <=0 -x 2 <=0 0 -x 2 <=0 -x 2 <=0 0 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) 3 2x 1 +3x 2 ≡0(mod12) 2x 1 +3x 2 ≡0(mod12) 3 x 1 ≡0(mod1) x 1 ≡0(mod1) 0 2x 1 +x 2 Ξ0(mod4) x 1 ≡0(mod1) x 1 ≡0(mod1) 0 2x 1 +x 2 Ξ0(mod4) x 2 ≡0(mod1) x 2 ≡0(mod1) 4 x 2 ≡0(mod1) x 2 ≡0(mod1) 4

IPs involve Lattices within Polytopes Objective = 30 2 / 3.. Objective = 30 2 / Objective = 28 Objective = Objective = Objective =

IPs involve Lattices within Polytopes Optimisation over polytopes give strongest (LP) bound on objective Optimisation over lattices give strongest congruence relation for objective Combined they give rank 1 cut for objective This may not be adequate

Lattices within Cones give Integer Monoids Lattices within Cones give Integer Monoids These are a fundamental structure for IP

Polyhedral and Non-Polyhedral Monoids The integer lattice within the polytope -2x + 7y >= 0 x – 3y >= 0 x – 3y >= 0 A Polyhedral Monoid y …… x x Projection: A non-polyhedral monoid (Generators 3 and 7) Projection: A non-polyhedral monoid (Generators 3 and 7) x.. x.. x x. x x. x x x……. x.. x.. x x. x x. x x x…….

Polyhedral and Non-Polyhedral Monoids …… x x Projection: A non-polyhedral monoid (Generators 3 and 7) Projection: A non-polyhedral monoid (Generators 3 and 7) x.. x.. x x. x x. x x x……. x.. x.. x x. x x. x x x……. Reverse Head Reverse Head x x. x x.. x.. x. x x. x x.. x.. x

Duality in LP and IP The Value Function of an LP Minimise x 2 Minimise x 2 subject to: 2x 1 + x 2 >= b 1 subject to: 2x 1 + x 2 >= b 1 5x 1 + 2x 2 <= b 2 5x 1 + 2x 2 <= b 2 -x 1 + x 2 >= b 3 -x 1 + x 2 >= b 3 x 1, x 2 >= 0 x 1, x 2 >= 0 Value Function of LP is Max( 5b 1 - 2b 2, 1/3( b 1 + 2b 3 ), b 3 ) Value Function of LP is Max( 5b 1 - 2b 2, 1/3( b 1 + 2b 3 ), b 3 ) If b 1 = 13, b 2 = 30, b 3 = 5 we have Max( 5, 7 2 /3, 5 ) = 7 2 /3 If b 1 = 13, b 2 = 30, b 3 = 5 we have Max( 5, 7 2 /3, 5 ) = 7 2 /3, Consistency Tester is Max( 2b 1 – b 2, -b 2, -b 2 + 2b 3 ) <= 0 giving Max( -4, -30, -20) <= 0. Consistency Tester is Max( 2b 1 – b 2, -b 2, -b 2 + 2b 3 ) <= 0 giving Max( -4, -30, -20) <= 0. (5, -2, 0), (1/3, 0, 2/3), (0, 0, 1) are vertices of Dual Polytope. (5, -2, 0), (1/3, 0, 2/3), (0, 0, 1) are vertices of Dual Polytope. They give marginal rates of change (shadow prices) of optimal objective with They give marginal rates of change (shadow prices) of optimal objective with respect to b 1, b 2, b 3. respect to b 1, b 2, b 3. (5, -2,, 0), (1/3, 0, 2/3), (0, 0, 1) are extreme rays of Dual Polytope. (5, -2,, 0), (1/3, 0, 2/3), (0, 0, 1) are extreme rays of Dual Polytope. What are the corresponding quantities for an IP ? What are the corresponding quantities for an IP ?

LP Solution Min x Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= 5 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= 0 x 2 x x x 1

IP Solution 9 Optimal IP Solution (2, 9)... Min x 2 9 Optimal IP Solution (2, 9)... Min x 2 c 3 st 2x 1 + x 2 >= 13 c 3 st 2x 1 + x 2 >= c1.. 5x 1 + 2x 2 <= c1.. 5x 1 + 2x 2 <= 30 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= 5 Optimal LP Solution (2 2 / 3, 7 2 / 3 ) -x 1 + x 2 >= c 2. x 1, x 2 >= c 2. x 1, x 2 >= 0 x 2 x x x 1

Duality in LP and IP The Value Function of an IP Minimise x 2 Minimise x 2 subject to: 2x 1 + x 2 >= b 1 subject to: 2x 1 + x 2 >= b 1 5x 1 + 2x 2 <= b 2 5x 1 + 2x 2 <= b 2 -x 1 + x 2 >= b 3 -x 1 + x 2 >= b 3 x 1, x 2 >= 0 and integer x 1, x 2 >= 0 and integer Value Function of IP is Value Function of IP is Max( 5b 1 - 2b 2, ┌ 1/3( b 1 + 2b 3 ) ┐, b 3, b ┌ 1/5 (-b ┌ 1/3(b 1 + 2b 3 ) ┐ ) ┐ ) Max( 5b 1 - 2b 2, ┌ 1/3( b 1 + 2b 3 ) ┐, b 3, b ┌ 1/5 (-b ┌ 1/3(b 1 + 2b 3 ) ┐ ) ┐ ) This is known as a Gomory Function. This is known as a Gomory Function. The component expressions are known as Chvάtal Functions. The component expressions are known as Chvάtal Functions. Consistency Tester same as for LP (in this example) Consistency Tester same as for LP (in this example)

Gomory and Chvátal Functions Max( 5b 1 -2b 2, ┌ 1/3(b 1 + 2b 3 ) ┐, b 3, b ┌ 1/5 (-b ┌ 1/3(b 1 + 2b 3 ) ┐ ) ┐ ) If b 1 =13, b 2 =30, b 3 =5 we have Max(5,8,5,9)=9 If b 1 =13, b 2 =30, b 3 =5 we have Max(5,8,5,9)=9 Chvátal Function b ┌ 1/5 (-b ┌ 1/3(b 1 + 2b 3 ) ┐ ) ┐ determines Chvátal Function b ┌ 1/5 (-b ┌ 1/3(b 1 + 2b 3 ) ┐ ) ┐ determines the optimum. the optimum. LP Relaxation is 19/15 b 1 - 2/5 b 2 +8/15 b 2 LP Relaxation is 19/15 b 1 - 2/5 b 2 +8/15 b 2 (19/15, -2/5, 8/15) is an interior point of dual polytope but (19/15, -2/5, 8/15) is an interior point of dual polytope but (5, -2, 0) and (1/3, 0, 2/3) are vertices of dual corresponding to possible (5, -2, 0) and (1/3, 0, 2/3) are vertices of dual corresponding to possible LP optima (for different b i ) LP optima (for different b i )

Why are valuations on discrete resources of interest ? Allocation of Fixed Costs Maximise ∑ j p i x i - f y Maximise ∑ j p i x i - f y st x i - D i y <= 0 for all I st x i - D i y <= 0 for all I y ε {0,1} depending on whether facility built. f is fixed cost. x i is level of service provided to i (up to level D i ) p i is unit profit to i. A ‘dual value’ v i on x i - D i y <= 0 would result in Maximise ∑ j (p i – v i ) x i - (f – (∑ j D i v i ) y Maximise ∑ j (p i – v i ) x i - (f – (∑ j D i v i ) y Ie an allocation of the fixed cost back to the ‘consumers’

A Representation for Chvátal Functions b 1 b 3 - b 2 b 1 b 3 - b Multiply and add on arcs 1 1 Divide and round up on nodes 2 2 Giving b ┌ 1/5( -b ┌ 1/3( b 1 + 2b 3 ) ┐ ) ┐ Giving b ┌ 1/5( -b ┌ 1/3( b 1 + 2b 3 ) ┐ ) ┐ LP Relaxation is 19/15 b 1 - 2/5 b 2 +8/15 b 3 LP Relaxation is 19/15 b 1 - 2/5 b 2 +8/15 b

Simplifications sometimes possible  ┌ 2 / 7 ┌ 7 / 3 n ┐ ┐ ≡ ┌ 2 / 3 n ┐  But ┌ 7 / 3 ┌ 2 / 7 n ┐ ┐ ≠ ┌ 2 / 3 n ┐ eg n = 1  ┌ 1 / 3 ┌ 5 / 6 n ┐ ┐ ≡ ┌ 5 / 18 n ┐  But ┌ 2 / 3 ┌ 5 / 6 n ┐ ┐ ≠ ┌ 5 / 9 n ┐ eg n = 5 Is there a Normal Form ? Is there a Normal Form ?

Properties of Chvátal Functions  They involve non-negative linear combinations (with possibly negative coefficients on the arguments) and nested integer round-up.  They obey the triangle inequality.  They are shift-periodic ie value is increased in cyclic pattern with increases in value of arguments.  They take the place of inequalities to define non-polyhedral integer monoids.

The Triangle Inequality  ┌ a ┐ + ┌ b ┐ >= ┌ a + b ┐  Hence of value in defining Discrete Metrics

A Shift Periodic Chvátal Function of one argument ┌ ½ ( x + 3 ┌ x /9 ┐ ) ┐ is (9, 6) Shift Periodic. 2 /3 is ‘long-run marginal value’  x  x

Polyhedral and Non-Polyhedral Monoids The integer lattice within the polytope -2x + 7y >= 0 x – 3y >= 0 x – 3y >= 0 A Polyhedral Monoid …… Projection: A Non-Polyhedral Monoid (Generators 3 and 7) Projection: A Non-Polyhedral Monoid (Generators 3 and 7) x.. x.. x x. x x. x x x ……. x.. x.. x x. x x. x x x ……. Defined by ┌ -x /3 ┐ + ┌ 2x /7 ┐ < = 0 Defined by ┌ -x /3 ┐ + ┌ 2x /7 ┐ < = 0

Finally  We should be Optimising Chvátal Functions over Integer Monoids

References  CE Blair and RG Jeroslow, The value function of an integer programme, Mathematical Programming 23(1982)  V Chvátal, Edmonds polytopes and a hierarchy of combinatorial problems, Discrete Mathematics 4(1973)  D.Kirby and HP Williams, Representing integral monoids by inequalities Journal of Combinatorial Mathematics and Combinatorial Computing 23 (1997)  F Rhodes and HP Williams Discrete subadditive functions as Gomory functions, Mathematical Proceedings of the Cambridge Philosophical Society 117 (1995)  HP Williams, A Duality Theorem for Linear Congruences, Discrete Applied Mathematics 7 (1984)  HP Williams, Constructing the value function for an integer linear programme over a cone, Computational Optimisation and Applications 6 (1996)  LA Wolsey, The b-hull of an integer programme, Discrete Applied Mathematics 3(1981)