1. 1 A classification approach for structure discovery in search spaces of combinatorial optimization problems Daniel Porumbel 1, 2, *, Jin Kao Hao 2, Pascale Kuntz 1 1 LINA CS. Laboratory University of Nantes, France 2 LERIA CS. Laboratory University of Angers, France *New a ffiliation from September 2010: LGI2A CS. Lab., Artois University, France

2. 2 Data Mining in Search Spaces Search Space: the set of all (potential) solutions of an optimization problem A heuristic Search Algorithm goes through this Search Space to (try to) find the optimal solution – Information on the structure of the Search Space is essential for heuristic guiding Main drawback: the Search Space is too large for complete enumeration or characterization Use Data Mining to discover hidden structures

3. 3 Towards guided optimization heuristics General objective: Adapt the search process (the heuristic) according to discovered search space properties Our approach: Consider a sample of the search space, i.e. a set of “best” candidate solutions Question: How are these solutions organized/structured ?

4. 4 Graph K-coloring problem A graph G=(V,E) is K-colorable if we can label its vertices with exactly K colors so that any two adjacent vertices have different labels (colors) Graph coloring problem Determine the minimal number K so that G is K-colorable : the chromatic number χ(G) NP-hard problem Our experimental framework : graph coloring problem

5. 5 Encoding of Coloring Solutions Solution C = a vector (c 1 |c 2 |…|c |V| |) of labels/colors Conflict number of C: number of edges with both ends of the same color K-coloring problem ≈ find a optimal solution C* minimizing the conflict number The problem is solved if C* has no conflicts Solution C=(3|1|1|1|2) Conflict number = 2 (V3-V2 and V3-V4)

6. 6 Search Space for K-coloring The set of all solutions has cardinal K |V| ~ a space of dimension |V| Neighborhood relation to pass from solution to solution: Pass from a solution C to neighboring C’ by changing just one color in a conflicting vertex of C Our approach for solving the coloring problem: Tabu Search

7. 7 Tabu Search [Glover and Laguna, Tabu Search, 1987] A local search moving from solution to solution by applying the neighborhood relation – It can move to a worser solution when there is no move that decreases the number of conflicts (no down move) Risk : to repeat a set of moves again and again => Tabu list = a list of temporary forbidden moves

8. 8 Classification of candidate solutions Question: given a set of best solutions, how are they distributed in the search space ? Our approach: 1. compute the distance (dissimilarity) between each two “best” solutions discovered by Tabu Search 2. analyze the solution distribution via the observed distance values one needs computing a distance with a low algorithmic complexity

9. 9 Fast Distance Calculation K-Coloring: the transfer distance between partitions [Régnier, Sur quelques aspects mathematiques des problemes de classification automatique, 1965] Classical complexity : Hungarian algorithm O(|V|+k 3 ) Las Vegas algorithm : O(|V|) time if the partitions are close enough (condition required for our problem) [Porumbel, Hao, Kuntz, An improved algorithm for computing the partition distance, Discrete Applied Mathematics, 2010]

10. 10 High Quality Solutions plotted via Multidimensional Scaling 350 best local minima represented via Multidimensional Scaling DIMACS graph G=dsjc250.5 with k = 27 These points form clusters that can be covered by spheres of small diameter.

11. 11 Tabu Search trajectory: MDS representation We consider a Tabu Search exploring the search space We launch it from a local optimum and we let it explore This figure plots the solutions of high-quality (not worse than the starting point) Intuitively, the visited high-quality colorings are grouped in clusters

12. 12 Consider a Tabu Search process exploring the search space: Record first 40.000 high-quality solutions and compute all distances (pairwise) between them Histogram: number of pairs of solutions for observed distance value We observe: Small distances: solutions in the same cluster Large distances: solutions in different clusters Trajectory of long Tabu Search processes

13. 13 Integrating learned information in the search process These analyses let us to the clustering hypothesis: The best candidate solutions are grouped into clusters that can be covered by spheres of specific diameter (10%|V|) Question: how to exploit such information to improve a search process? Answer: consider a sphere-based search space organization

14. 14 Integrating learned information in the search process These analyses let us to the clustering hypothesis: The best candidate solutions are grouped into clusters that can be covered by spheres of specific diameter (10%|V|) Question: how to exploit such information to improve a Search process? Answer:consider a sphere-based search space organization

15. 15 Numerical Results Integrating learned information (e.g. clustering information) helped a basic local search to reach competitive results We proposed one of the first local search algorithms that can compete with complex population-based heuristics [Porumbel, Hao & Kuntz, A Search Space Cartography for Guiding Graph Coloring Heuristics, Computers & OR, 2010 ] First coloring with k=223 colors for the well-studied DIMACS graph dsjc1000.9

16. 16 Detailed Numerical Results

17. 17 Conclusions Clustering information can be very useful in guiding any search algorithm We also employed the clustering hypothesis in an evolutionary coloring approach Such techniques can be used for any combinatorial optimization problem – given a distance function that can rapidly computed More advanced techniques can be used to improve our knowledge of search spaces Using learning in optimization seems a promising direction (e.g. the Learning and Intelligent Optimization)