1. Quality of LP-based Approximations for Highly Combinatorial Problems Lucian Leahu and Carla Gomes Computer Science Department Cornell University

2. Motivation Increasing interest in combining Constraint Satisfaction Problem (CSP) formulations and Linear Programming (LP) based techniques for solving hard computational problems. Successful results for solving problems that are a mixture of linear constraints – where LP excels – and combinatorial constraints – where CSP excels. In a purely combinatorial setting, surprisingly difficult to effectively integrate LP- and CSP-based techniques

3. Goal Study and characterize the quality of LP based heuristics for highly combinatorial problems.

4. Research Questions Is the quality of LP-based Approximations related to the structure of the problem? (Typical case, rather than worst case) How is the quality of LP-based Approximations influenced by different formulations of the problem? Does the LP relaxation provide a global perspective of the search space? Is the LP relaxation good as a heuristic to guide complete solvers?

5. Outline A highly combinatorial search problem --- quasigroup completion problem (QCP) LP-based formulations for QCP Assignment based formulation Packing formulation Quality of LP based approximations LP as a global search heuristic Conclusions

6. Latin Squares or Quasigroups Given an N X N matrix, and given N colors, a quasigroup of order N is a a colored matrix, such that: all cells are colored. each color occurs exactly once in each row. each color occurs exactly once in each column. Quasigroup or Latin Square (Order 4)

7. Latin Squares/Quasigroups Completion Problem Given a partial assignment of colors (10 colors in this case), can the partial Latin Square be completed so we obtain a full square?

8. Latin Squares/Quasigroups Completion Problem Given a partial assignment of colors (10 colors in this case), can the partial Latin Square be completed so we obtain a full square? Example: Structure of this problem characterizes several real-world applications: e.g., Timetabling, sports scheduling, rostering, routing, etc.

9. Quasigroup with Holes (QWH) Given a full quasigroup, “punch” holes into it QWH is NP-Hard. 32% holes Advantage: we know the optimal value.

10. LP-based formulations for QCP

11. Assignment Formulation Variables - Max number of colored cells s.t. at most one color per cell: a color appears at most once per row a color appears at most once per column

12. Assignment Formulation

13. New Phase Transition Phenomenon: Integrality of LP Note: standard phase transition curves are w.r.t existence of solution) Sudden phase Transition in solution integrality of LP relaxation and it coincides with the hardest area holes/n^1.55 No of backtracks Max value of LP Relaxation

14. Packing formulation Max number of colored cells in the selected patterns s.t. one pattern per family a cell is covered at most by one pattern Families of patterns (partial patterns are not shown)

15. Packing formulation

16. Previous Results 0.5 approximation based on Assignment formulation – Kumar et al. – 1999 (1-1/e ≈ 0.63) approximation based on Packing formulation – Gomes, Regis, Shmoys – 2003 Use of LP to select variables and values and to prune search trees – Refalo et al. – 1999, 2000 No typical case results on the quality of LP based approximation

17. Quality of LP-based Approximations

18. Approximation Schemes LP Formulations: Assignment formulation; Packing formulation; Approximation scheme: solve the LP relaxation and interpret the resulting solution as a probability distribution; Order for Variable Setting Uniformly at Random Greedy Random Greedy Deterministic Increasing greediness

19. Uniformly at Random

21. holes/n^1.55 % of colored holes

22. Uniformly Random - Comparison Drop in quality of approximation as we enter the critically constrained area The quality stabilizes in the under constrained area Random LP Packing does better, since the corresponding LP relaxation is stronger Random LP Packing is a 1 – 1/e≈0.63 approximation, while LP assignment ½ approximation. % of colored holes holes/n^1.55

23. Greedy Random

25. % of colored holes holes/n^1.55

26. Greedy Random - Comparison Drop in quality of approximation as we enter the critically constrained area The quality increases in the under constrained area --- info provided by LP is used in a more greedy way (more valuable); forward checking also improves quality. Random LP Packing does slightly worse, since it optimizes an entire matching % of colored holes holes/n^1.55

27. Greedy Deterministic

29. Greedy Deterministic - Comparison Drop in quality of approximation as we enter the critically constrained area The quality increases in the under constrained area --- info provided by LP is used in a more greedy way and deterministically (more valuable); forward checking also improves quality. Random LP Packing does slightly worse, since it is less greedy (sets an entire matching), doesn’t use as much lookahead % of colored holes holes/n^1.55

30. Comparison with Pure Random Strategy % of colored holes holes/n^1.55

31. LP as a Global Search Heuristic Can LP guide complete solvers? Use an LP relaxation to set a certain percent of variables (the highest values) Run a complete solver on the resulting instances and check if it is still completable (we start with a PLS that is completable)

32. LP as a Global Search Heuristic - Results 1 hole % of satisfiable instances holes/n^1.55 5% holes/n^1.55 % of satisfiable instances

33. Conclusions Quality of approximation is directly correlated with phase transition phenomenon – closely related to constrainedness regions of the problem (sharp decrease in the critical region)  New phase transition in the integrality of the LP relaxation solution Typical case analysis – although theoretical bounds for LP packing are stronger, the empirical results for enhanced versions of approximations (with forward-checking) seem to indicate that LP approximations based on the assignment formulation are better (but difficult to analyze theoretically) LP can provide useful high level guidance + should be combined with random restart strategies to recover from potential mistakes made at the top of the tree

34. Quality of LP-based Approximations for Highly Combinatorial Problems Lucian Leahu and Carla Gomes Computer Science Department Cornell University