1. A Polyhedral Approach to Cardinality Constrained Optimization Ismael Regis de Farias Jr. and Ming Zhao University at Buffalo, SUNY

2. Summary Problem definition Relation to previous work Simple bound inequalities Further research

3. Problem Definition Given c 1  n, A m  n, b m  1, and l n  1, u n  1 ≥ 0, find x n  1 that: maximizes cx subject to Ax  b, − l ≤ x ≤ u, and at most k variables are nonzero

4. Motivation Portfolio selection Feature selection in data mining

5. Polyhedral approach Derive within a branch-and-cut scheme strong inequalities valid for: P i = conv {x  R n :  j  N a ij x j  b i, − l ≤ x ≤ u, and at most k variables are nonzero}, i  {1, …, m}, to use as cutting planes in the branch-and-cut

6. Previous work Bienstock (1996): critical set inequalities de Farias and Nemhauser (2003): cover inequalities. However, the present case is more general and the polyhedral structure is much richer …

7. Example Let P = conv {x  [ − 1, 1] 6 : 6 x 1 + 4 x 2 + 3 x 3 + 2 x 4 + x 5 + x 6  6 and at most 3 variables are nonzero}. The following inequalities define facets of P: 6 x 1 + 4 x 2 + 3 x 3 + 2 x 4 + x 5 + x 6  6 4 x 2 + 3 x 3 + 2 x 4 + x 5 + x 6  6 4 x 2 + 3 x 3 + 2 x 4 + x 5  6 4 x 2 + 3 x 3 + 2 x 4 + x 6  6 4 x 2 + 2 x 4 + x 5 + x 6  6 4 x 2 + 2 x 4 + x 6  6 4 x 2 + 2 x 4 + x 5  6

8. To take advantage of previous work … first, we scale and translate the variables, i.e. P = conv {x  [0, 1] n :  j  N a j x j  b and x j  β j, j  N, for at most k variables}, and second, we consider the pieces of P

9. The pieces are defined as follows … Proposition Let W  N, X W = {x  R n : x j  β j  j  W and x j ≤ β j  j  N − W}, and P W = P ∩ X W. Then, P W = conv (S ∩ X W ), where S = {x  [0, 1] n :  j  N a j x j  b and x j  β j, j  N, for at most k variables}.

10. For each piece … i.e. for a given W, we change the variables as: y j ← (x j – β j ) / (1 – β j ), j  W y j ← ( β j – x j ) / β j, j  N − W

11. Example P = conv {x  [0, 1] 2 : 6 x 1 + 4 x 2  7, and x 1 = ½ or x 2 = ½}. ½ ½ 1 1 x1x1 x2x2 6 x 1 + 4 x 2 = 7

12. Example P = conv {x  [0, 1] 2 : 6 x 1 + 4 x 2  7, and x 1 = ½ or x 2 = ½}. ½ ½ 1 1 x1x1 x2x2 PNPN P {1} P {2} PP

13. Example P = conv {x  [0, 1] 2 : 6 x 1 + 4 x 2  7, and x 1 = ½ or x 2 = ½}. ½ ½ 1 1 x1x1 x2x2 3 y 1 + 2 y 2  2−3 y 1 + 2 y 2  2 3 y 1 − 2 y 2  2 −3 y 1 − 2 y 2  2 at most 1 nonzero

14. When a j  0 and b > 0 … Proposition The inequality  j  N x j  k is facet-defining iff a n − k + …+ a n − 1  b and a 1 + a n − k+2 + …+ a n  b. Proposition When a n − k + …+ a n − 1  b and a 1 + a n − k+2 + …+ a n > b, the inequality a 1 x 1 +  2≤j≤n − k − 1 max {a j, Δ} x j +Δ  n − k≤j≤n x j ≤ k Δ defines a facet of P, where Δ = (b −  n − k − 2≤i≤n a i ).

15. It then follows that … Proposition The inequality:  j  W (x j – β j )/(1 – β j )–  j  N − W (x j – β j )/ β j  k is valid  W  N, and it is facet-defining “under certain conditions”.

16. In the same way … Proposition The inequality: a 1 (x 1 – β 1 )/(1 – β 1 ) +  2≤j≤n − k − 1, j  W max {a j, Δ}(x j – β j )/(1 – β j ) +  2≤j≤n − k − 1, j  N − W max {a j, Δ} (x j – β j )/ β j + Δ  n − k≤j≤n, j  W (x j – β j )/(1 – β j ) + Δ  n − k≤j≤n, j  N − W (x j – β j )/ β j ≤k Δ defines a facet of P “under certain conditions”.

17. Example P = conv {x  [0, 1] 2 : 6 x 1 + 4 x 2  7, and x 1 = ½ or x 2 = ½}. ½ 1 1 x1x1 x2x2 x 1 + x 2  3/2 x 1 − x 2  1/2 x 1 + x 2 ≥ 1/2 (y 1 + y 2  1 and 3 y 1 + 2 y 2  2) (y 1 + y 2  1 and 3 y 1 − 2 y 2  2) −x 1 + x 2  1/2 (y 1 + y 2  1 and −3 y 1 − 2 y 2  2) (y 1 + y 2  1 and −3 y 1 + 2 y 2  2)

18. Critical sets and covers By fixing, at 0 or 1, variables with positive or negative coefficients, we can obtain implied critical sets or cover inequalities that define facets in the projected polytope. Then, by lifting the fixed variables, we obtain strong inequalities valid for P

19. Example Let P = conv {x  [0,1] 5 : 6x 1 + 4x 2 − 3x 3 − 2x 4 + x 5  6 and at most 2 variables are positive}. Fix x 3 = 1 and x 4 = 0. The inequality: 6x 1 + 4x 2 + 3x 5  9 defines a facet of P ∩ {x  [0,1] 5 : x 3 = 1 and x 4 = 0}.

20. Simple bound inequalities Let P = conv {x  [0,1] 4 : 6x 1 − 4x 2 + 3x 3 − x 4  3 and at most 2 variables are positive}. Fix x 3 = x 4 = 0. Then, x 1  1 defines a facet of P ∩ {x  [0,1] 4 : x 3 = x 4 = 0}. Lifting with respect to x 4, we obtain x 1 + α x 4 ≤ 1, which gives α = ⅓. Lifting now with respect to x 3, we obtain 3x 1 + α x 3 + x 4 ≤ 3, which gives α = 2, and so 3x 1 + 2 x 3 + x 4 ≤ 3.

21. Additional results Two families of lifted cover inequalities Two families of inequalities derived from simple bounds Necessary and sufficient condition for “pieces of a facet” to be a facet

22. Further Research Separation routines and computational testing Inequalities derived from intersection of knapsacks Special results for feature selection in data mining